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1.  Contract  Objectives: 


Design,  fabricate,  test  and  deliver  one  Miniature  Imaging  Photometer 
(MIP),  in  accordance  with  Keo  Consultants  Technical  Proposal  entitled  Miniatuie 
Imaging  Photometer  for  Aurora  and  Airglow,  dated  89JAN31,  as  amended  89APR19. 

2.  Description  of  Progress: 

(a)  Optical: 

Quarterly  Reports  KEO-1  to  KEO-4  described  design  decisions  made 
during  the  first  quarter  and  progress  in  implementing  those  designs  during  the  second 
through  4th  quarters.  Current  status  of  design  and  fabrication  of  various  optical 
components  of  the  system  are  summarized  in  the  following: 

(i)  All-sky  Lens:  See  Figure  1  for  optical  schematic  of  the  telecentric  design. 
The  telecentric  elements  have  been  procured  (two  plano-convex  lenses,  with  anti¬ 
reflection  coatings  all  surfaces).  This  part  of  the  design  has  been  bench  tested  and 
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(ii)  Filters  and  Filter  Wheel:  Five-position,  1.75M  dia.  filters.  The  filter  wheel 
has  been  fabricated  and  successfully  tested. 

A  set  of  five  filters  has  been  ordered,  at  the  following  wavelengths: 

427.8  nm 

455.4  nm 

555.7  nm 

630.0  nm 

777.4  nm 

(iii)  Re-imaging  Optics:  The  assembly  as  designed  by  Kidger  Optics  uses  two 
standard  (Canon)  camera  lenses  and  one  plano-convex  lens  (see  Figure  2).  Optical 
bench  testing  of  the  Kidger  Optics  design  revealed  a  vignetting  problem  with  the 
Canon  85mm  lens.  This  was  traced  to  the  fact  that  the  commercial  Canon  85mm/F1 .2 
lens  measures  to  be  only  about  FI. 3,  and  this  difference  is  sufficient  to  cause  the 
problem.  Kidger  is  reconsidering  the  design  in  light  of  this  problem,  but  to  date  has  not 
been  able  to  come  up  with  a  solution  that  still  uses  commercially  available  camera 
lenses.  In  the  meantime,  a  scaled  down  version  of  a  previous  larger-format  design 
developed  by  Keo  is  being  considered  as  an  alternative.  Bench  testing  indicates  good 
performance  (no  vignetting)  but  there  is  significant  curvature  of  the  image  field.  A 
range  of  piano  concave  lenses  (to  act  as  a  field  flattener)  were  procured  and  bench 
tested;  curvature  can  be  considerably  corrected,  but  additional  astigmatism  is 
introduced.  The  design  is  currently  being  further  investigated  using  computer  design 
software. 

(iv)  Calibration  Capability:  A  (permanent)  45  deg.  partially  reflecting  mirror 
(90%/10%)  will  be  used  to  direct  calibration  charts  into  the  optics  train  (after  the  filter 
plane),  see  Figure  3.  Options  (manually  switchable)  will  include  resolution  chart,  gray 
scale  chart,  series  of  relative  calibration  targets,  etc.  This  mirror  has  been  included  in 
the  design,  but  as  yet  has  not  been  procured.  (These  elements  will  be  ordered  only 
after  satisfactory  testing  of  the  reimaging  optics.) 

(v)  Shutter:  A  large  (2.5")  aperture  shutter,  positioned  near  an 
aperture  plane  (so  that  shutter  open/close  characteristics  do  not  affect  relative 
exposure  over  the  image  area)  is  for  sky  exposure  control.  The  shutter  has  been 
procured,  modified  and  mechanically  tested.  Low-force  microswitches  were  installed 
to  confirm  shutter  open  status,  and  a  photocell  also  installed  to  allow  premonitoring  of 
input  light  intensities.  This  latter  feature  will  be  used  both  as  protection  and  as  a 
means  of  judging  optimum  exposure  and  high  voltage  setting. 
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(vi)  Fabry-Perot  Option:  Provision  for  insertion  of  a  Queensgate 
Fabry-Perot  interferometer  cell  has  been  included  in  the  design.  This  option 
however  will  not  be  included  in  this  instrument  under  this  Contract. 

(vii)  Image  Intensifier  Lens:  Has  been  defined  to  be  a  Canon  50mm/F1.2 
(and  has  been  procured). 

(viii)  Image  Intensifier:  Design  is  to  allow  interchangeability  of  25mm  types. 
Intensifiers  with  near  uv  sensitivity  (1st  Gen  +  Proximity  hybrid),  visible  (normal  Gen  II) 
or  infrared  sensitive  (Gen  III)  may  be  used.  To  minimize  thermal  noise  (especially 
important  if  Gen  III  type  used),  the  image  intensifier  cathode  is  to  be  cooled  to 
approximately  0°.C.  The  phosphor  is  kept  at  ambient,  as  cooling  increases  decay/lag 
times  appreciably.  We  discovered  recently  that  a  25mm  Gen  III  tube  is  not  available  in 
inverter  form;  an  18mm  version  is  available,  but  is  of  different  mechanical 
configuration.  A  25mm  Gen  III  proximity  tube  can  be  obtained  from  ITT  (expensive), 
and  so  could  be  integrated  with  a  Gen  I  tube  to  form  an  IR-hybtid,  if  future  research 
plans  call  for  near-IR  imaging. 

(ix)  Image  Intensifier  Cooler:  A  prototype  thermoelectric  cooler  was  built  in 
the  first  quarter  (by  Products  for  Research),  and  testing  indicated  satisfactory 
performance.  A  final  version  was  designed  by  Keo,  which  included  provision  for  dry 
nitrogen  flushing,  larger  area  heat  dissipation  fins,  RTD  temperature  readout,  and 
appropriate  mounting  provisions.  This  cooler  has  now  been  procured,  and  testing 
indicates  performance  to  specifications.  The  cooler  next  has  to  be  mounted  in  a  fan- 
cooled  housing. 

(x)  Relay  Optics:  A  fast,  high-resolution  relay  lens  pair  consisting  of 
Rodenstock  100mm. FI. 4  +  42mm/F0.7  will  be  used,  reducing  the  image  diameter  to 
the  10mm  required  to  fit  on  the  CCD.  These  lens  components  have  been  procured, 
and  a  design  has  been  completed  for  mounting  a  large  aperture  shutter  between  the 
lenses  (see  xi.  below). 

(xi)  CCD  Shutter:  A  second  large  aperture  shutter  (see  (v)  above)  between 
the  relay  lenses  will  ensure  the  CCD  is  kept  dark  (from  image  intensifier  dark  noise  or 
phosphor  lag)  during  filter  changes  or  between  any  two  exposures. 

(xii)  Image  Intensifier  Output  Monitor:  Part  of  the  beam  between  relay  lenses 
will  be  sampled  with  a  photoconductive  cell.  This  will  be  used  to  adjust  image 
intensifier  gain  under  computer  control  so  that  the  image  intensifier  is  kept  in  its  linear 
gain  region  (before  AGC  switches  in),  and  to  assist  in  setting  CCD  exposure  time.  This 
has  been  incorporated  into  the  shutter,  and  will  be  tested  next  quarter. 
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(b)  Detector  -  CCD  and  Housing: 


The  first  quarter  of  this  contract  involved  an  in-depth  review  of  present  CCD 
technology  to  decide  which  CCD  we  should  choose  for  the  MIP.  After  much 
consideration,  we  decided  that  we  wanted  to  take  advantage  of  the  new  MPP 
technology.  Multi-Pin-Phased  (MPP)  CCD's  have  a  boron  implant  that  when  driven 
with  the  proper  phasing,  have  a  dark  noise  of  1  /20th  of  a  similiar  CCD  without  the 
boron  implant.  This  technology  was  developed  jointly  by  Photometries  and  Ford 
Aerospace. 

Photometries  developed  a  512x512  chip  with  an  18  urn  pixel  size.  SAIC 
subsequently  developed  a  1024x1024  chip  with  the  same  properties  as  the 
Photometries  chip.  Both  chips  were  carefully  evaluated  for  our  application  and  an 
analysis  is  included  in  Appendix  1  of  this  report.  It  was  determined  that  the 
Photometries  chip  best  suited  our  needs  for  the  MIP  in  terms  of  matching  resolution, 
dynamic  range,  and  cost.  The  significantly  reduced  thermal  noise  will  allow  long 
exposures  and  better  S/N  in  low  intensity  situations  (eg.  heater  experiments,  Fabry- 
Perot  applications). 

Initial  design  for  the  camera  housing  considered  using  standard  Photometries 
circuit  boards,  and  investigating  how  they  could  be  miniaturized  for  our  application. 
Their  air-cooled  CH200  system  was  selected,  and  a  repackaging  scheme  for  their 
camera  electronics  was  developed. 

From  the  Photometries  schematics  we  re-designed  their  interface  boards  to 
physically  fit  into  our  mechanical  camera  configuration.  An  analysis  of  the  benefits  of  a 
surface-mount  implementation  was  performed  and  it  was  found  that  while  saving 
about  30%  in  physical  size,  both  the  cost  and  serviceability  made  it  impractical.  Finally 
a  two  board  set  was  decided  upon  that  would  fit  into  the  MIP  camera  head 
configuration. 

Photometries  subsequently  formed  a  splinter  company  called  Advanced 
Technologies  headed  by  Dr.  Gary  Simms.  Advanced  Technologies  (AT)  was 
developing  a  small  camera  head,  and  interface  board  set  for  Marine  Imaging  Systems 
that  closely  met  our  needs.  In  October  89  we  met  with  Dr.  Simms  and  agreed  on  a 
product  specification  and  in  December  89  he  supplied  us  with  a  cost  estimate.  We 
decided  to  use  this  camera  head/electronics  implemented  with  the  Photometries  MPP 
CCD  and  a  generalized  12  bit  parallel  interface  that  could  either  be  interfaced  to  the 
Macintosh  computer  or  to  one  of  the  available  IBM  PC  based  Image  Processing 
boards. 
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A  specially  designed  head  to  accommodate  our  fast  relay  lens,  and  with 
enhanced  cooling  fins  and  connector  configuration  to  suit  our  mechanical 
requirements,  has  now  been  designed  by  Photometries. 

A  firm  quote  was  finally  received  from  AT  in  May  90  for  the  cost  of  the  system 
excluding  the  custom  interface  board  that  we  are  designing  to  control  all  our  camera 
electronics.  Progress  has  been  very  slow  with  AT  as  they  had  to  finish  up  their  product 
with  Marine  Imaging  Systems  first.  Complete  interface  specifications  were  sent  to 
Doug  Donahue  (contact  engineer  at  AT)  along  with  a  set  of  circuit  schematics 
designed  specifically  for  our  camera  system,  functional  diagrams  for  all  the  MIP 
components,  and  the  planned  physical  layout  of  the  system. 

Advanced  Technologies  has  finally  started  work  on  our  project  the  last  week  of 
June;  they  plan  to  finalize  the  design  by  the  end  of  July  and  present  it  for  our  final 
review,  and  delivery  is  slated  for  the  first  week  in  October.  While  this  is  much  later  than 
we  would  like  (AT  first  estimated  delivery  at  March  90),  this  should  give  us  sufficient 
time  to  integrate  the  camera  head  and  electronics  into  the  MIP  before  late  winter  (Feb  - 
Mar.,  1991)  for  field  testing  (probably  in  Alaska). 

(c)  MIP  Electronics  and  Control  System: 

The  MIP  control  system  consists  mainly  of  the  filter  wheel  housing/electronics, 
shutter  controls,  intensifier  cooler/electronics,  temperature  measurement/control,  and 
manual  control  panel.  It  was  decided  to  develop  a  set  of  electronics  for  the  MIP  that 
was  compact  and  fully  implemented  in  the  actual  MIP  unit.  In  addition,  it  was  decided 
that  for  both  compactness  and  convenience  all  manual  controls  would  actually  be 
located  in  the  camera  itself  rather  than  the  traditional  rackmount  unit  with  the 
computer.  This  has  the  advantage  that  work  done  to  control  the  camera  manually  is 
usually  done  when  the  computer  is  down,  or  is  not  conveniently  located  (as  all  aspects 
of  the  MIP  can  be  controlled  from  the  computer  as  well).  In  addition,  have  all  these 
components  located  in  the  camera  housing  provides  less  cabling,  connectors,  and 
packaging.  These  will  improve  the  reliability  and  portability  of  the  instrument. 

(i)  Filter  Wheel  Control: 

The  filter  wheel  has  been  built  and  the  interface  electronics  has  been 
prototyped  and  a  lab  version  of  the  electronics  is  presently  working.  The  circuit  uses 
an  intelligent  stepper  motor  controller  that  runs  self-contained  with  a  memory  chip. 
This  has  the  advantage  of  small,  intelligent  and  extremely  flexible  control  system  that 
can  be  easily  modified  to  meet  the  needs  of  any  changes  in  the  future.  Small,  powerful 
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electronics  components  such  as  single-package  1.5A  Stepper  Motor  Drivers,  and  TO- 
220  package  high  Wattage  resistors  have  been  selected  to  minimize  space  for  the 
camera. 

This  control  system  has  been  found  to  work  very  well  and  is  a  marked 
improvement  on  the  two  previous  control  systems,  as  the  acceleration  and 
deceleration  of  the  filter  wheel  is  controlled  by  a  mathematical  algorithm  to  ensure 
smooth  operation  and  a  minimum  chance  of  loosing  steps.  An  automatic  homing 
function  automatically  sets  the  final  filter  destination  every  move  to  ensure  consistent 
filter  placement.  Final  stepper  motor  voltages,  dropping  resistors,  and  acceleration 
algorithms  have  to  be  determined  once  the  filter  wheel  is  anodized,  and  filters  are 
added  to  the  system. 

(ii)  Shutter  Control: 

Large  aperture  Uniblitz  shutters  were  chosen  for  the  MIP  to  mount  in  an 
aperture  plane  so  as  eliminate  the  effect  of  vignetting  due  to  a  finite  shutter  velocity. 
Because  these  large  shutters  require  a  higher  power  control  circuitry,  much  effort  has 
gone  into  the  design  of  a  powerful  but  compact  shutter  driver  circuit.  First,  the  drivers 
supplied  by  Uniblitz  were  evaluated  and  found  to  be  adequate,  but  not  very  cost  or 
size  effective. 

The  Advanced  Technologies  shutter  circuits  were  finally  supplied  to  us  in  May, 
1990  for  evaluation  and  they  proved  to  be  inadequate.  Modifications  have  been  made 
to  the  circuit  so  that  they  could  drive  our  shutters,  however  they  involve  large 
capacitors  to  provide  discharging  current  to  open  the  shutters.  A  design  was 
developed  by  KEO  to  use  advanced  supply  drivers  and  a  CMOS  solenoid  driver  that 
provides  ample  starting  current,  and  is  minimized  for  space.  A  dual  shutter  driver  has 
been  built  for  a  lab  prototype  and  has  been  found  to  work  well  with  our  modified 
Uniblitz  shutters.  Final  testing  will  be  done,  when  the  shutters  are  anodized  and 
assembled.  Limit  switches  are  included  in  the  shutter  design  to  provide  a  TTL  output  to 
the  interface  electronics  indicating  that  the  shutter  is  completely  open.  This  will  be  a 
very  useful  feature  in  the  field  for  diagnostics. 

(ni)  Photoconductive  Cell: 

A  Clairox  PL709L  Photoconductive  cell  was  evaluated  to  measure  the  overall 
brightness  of  the  image  intensifier.  It  was  determined  that  this  will  be  the  easiest  way  to 
measure  the  image  intensifier  and  check  for  saturation.  The  photoconductive  cell  will 
be  interfaced  through  the  camera  head  into  the  ADC  port  E  of  the  68HC1 1  MPU  on  the 
Advanced  Technologies  Controller  Board.  A  smaller,  lower  resistance  version  of  the 
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709  has  been  on  order  from  Senisys  for  evaluation  to  be  located  in  the  Shutter 
housing.  We  expect  to  receive  this  during  the  middle  of  July  for  final  evaluation. 

(iv)  Intensifier  Electronics/Housing: 

The  Image  Intensifier  Cooler  has  been  built  and  tested.  It  will  require 
approximately  6  Volts  unregulated  DC  to  operate.  A  final  power  supply  will  be  chosen 
once  we  get  the  final  specifications  of  the  TEC  for  the  CCD  camera  head.  The  image 
intensifier  power  supply  and  gain  control  has  been  designed  and  is  a  modified  version 
of  KEO's  last  control  circuit.  It  has  not  yet  been  prototyped  as  we  have  a  high  degree  of 
confidence  that  the  circuit  will  work.  A  prototype  will  be  built  late  July  or  early  August. 

(v)  Temperature  Measurement  and  Control: 

A  somewhat  complicated  temperature  measurement  scheme  was  developed  to 
allow  measurement  of  all  temperatures  both  at  the  manual  control  panel,  and  at  the 
computer  simultaneously,  using  only  one  RTD  for  each  temperature.  Two  Omega 
Temperature  Readout/Controller  Panel  Meters  have  been  purchased  to  be  mounted  in 
the  control  panel  of  the  MIP.  These  are  very  small  packages  and  are  ideal  for  our 
application.  A  four  position  4PDT  interlocked  switching  scheme  will  switch  excitation 
current  for  the  RTD's  between  the  Omega  supply  and  the  supply  located  on  the  KEO 
interface  board.  RTD  instrumentation  amplifiers  will  then  amplify  the  RTD  signal  end 
calibrate  the  signal  within  the  temperature  range  of  -50C  to  78C.  These  signals  will  be 
inputted  into  Port  E  (ADC)  of  the  68HC11  and  digitized  to  eight  bits  giving  us  a 
resolution  of  1/2  degree.  This  temperature  range  should  adequately  cover  the  ranges 
of  temperature  encountered  in  the  MIP.  The  RTD  interface  has  been  designed  based 
on  the  Advanced  Technologies  design  for  Marine  Imaging  Systems  to  ensure 
compatibility  with  the  AT  interface. 

(vi)  Interfacing  to  the  68HC1 1  MPU: 

A  simple  but  flexible  interface  to  the  MPU  has  been  designed  to  implement  the 
KEO  control  system  into  the  Advanced  Technologies  board  set.  This  is  presently  going 
under  design  review  at  AT.  This  will  provide  us  with  all  the  signals  necessary  to 
implement  the  present  control  design.  A  controller  board  will  be  sent  to  us  from  AT  in 
August  for  evaluation  of  the  design. 


(d)  Image  Acquisition  Storage  and  Display: 
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A  detailed  study  of  the  current  technology  and  trends  in  computers  and  image 
processing  was  undertaken  to  determine  the  best  scheme  for  implementation  in  the  MIP. 
This  analysis  of  the  present  image  processing  and  computational  needs  and 
anticipated  needs  in  the  near  future  is  presented  in  Appendix  2. 

To  maximize  the  benefit  of  the  MIP  in  the  field,  the  scientist  will  require  a  lame 
degree  of  flexibility  in  use,  and  as  much  processing  power  as  possible.  It  was  decided 
that  to  spend  a  great  deal  of  effort  to  make  a  truly  small  instrument  at  the  sacrifice  of  the 
power  that  is  available  today  packaged  in  the  standard  personal  computer  was  counter¬ 
productive. 

The  MIP  as  presently  envisioned  will  have  one  monitor  for  use  in  the  field  that  will 
display  both  control  information  and  image  information  with  the  option  of  using  two 
(possibly  higher  resolution)  monitors  in  the  lab  for  subsequent  detailed  analysis  of  data. 
The  package  will  be  a  standard  PC  desktop  unit  that  will  contain  all  the  necessary 
circuitry  to  interface  directly  to  the  camera  body.  This  will  provide  a  compact  and 
extremely  powerful  system  that  will  many  times  outperform  capabilities  of  the  present 
ASIP  systems,  as  well  as  outperform  the  data  processing  capabilities  of  the  computers 
GL  currently  uses  to  analyze  the  data. 

The  envisioned  MIP  would  no  longer  be  just  a  field  instrument,  but  rather  a  small, 
powerful  image  processing  work  station  set  up  to  either  take  automated  ASIP  images, 
and  to  additionally  have  the  capability  to  analyze  the  images  and  develop  new  ways  to 
configure  the  instrument  and  data  taking  routine  while  in  the  field. 

To  do  this,  three  different  approaches  have  been  looked  into  in  detail.  These 
would  be  implemented  on  either  the  Mac  llci  using  a  custom  interface  board  being 
developed  by  Advanced  Technologies,  or  the  IBM  PC  compatibles  using  either  the 
MATROX  Image  Series  Board  Set,  or  the  Image  Technology  Advanced  Frame  Grabber 
board  set.  Both  companies  have  been  visited  in  the  last  two  months.  Complete  sets  of 
documentation  from  the  manufacturers  were  evaluated  and  several  reports  were  writtr  n 
to  help  choose  which  of  the  systems  best  meets  our  needs  (  see  Appendix  3). 

A  final  decision  will  be  made  in  July  after  an  engineering  analysis  of  the  ITI  board 
has  been  completed  at  the  factory,  and  after  final  consultations  with  Air  Force  scientists. 
Once  this  decision  is  made,  the  boards  and  the  computer  will  be  acquired  to  start 
development  of  the  computer  system. 

Some  thought  has  gone  into  whether  the  MIP  should  be  packaged  as  an 
industrial  rack  mount  system  or  just  as  a  normal  desktop  computer.  It  was  decided  that  it 
would  be  best  to  use  the  standard  desktop  computer,  as  they  are  about  the  third  the 
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cost,  have  been  proven  to  work  reliably  in  the  field,  and  provide  a  much  more  flexible 
and  portable  working  system  for  cramped  field  sites.  In  addition,  replacement  boards  are 
readily  available  and  inexpensive.  Also,  using  standard  PC  products  ensures  that  we 
will  easily  be  able  to  upgrade  to  take  advantage  of  quickly  evolving  technologies. 


(e)  Software  Development: 

Early  work  in  1990  focussed  on  becoming  proficient  in  C  and  programming  in 
an  event-oriented  structure  using  the  Macintosh  toolbox  routines.  No  further  software 
development  has  been  pursued  as  the  final  operating  system  has  yet  to  be 
determined.  However,  operating  systems  for  the  IBM  AT  environment  were 
investigated  and  the  Windows  3.0  operating  system  was  chosen  as  the  first  operating 
system  (using  DOS  4.0)  for  the  IBM  environment.  This  would  prove  compatible  with 
existing  operating  systems  at  the  Geophysics  Laboratories,  and  would  provide  upward 
compatibility  into  true  32-bit  operating  systems  (OS/2)  and  true  32-bit  code  (WATCOM 
8.0/386).  Recently  we  have  been  familiarizing  with  the  capabilities  of  the  Windows 
software. 


Appendix  1 


Discussion  of  CCD  Selection 


KEO  Consultants  8/1/89 


An  in-depth  summary  of  the  different  characteristics  of  the  CCD's  we  are 
considering  is  necessary  at  this  point  in  order  to  make  a  decision  for  the  design  of  the 
Miniature  Imaging  Photometer.  We  have  made  a  commitment  to  trying  to  use  CCD's 
with  multi-pinned  phased  (MPP)  capability.  For  our  purposes,  this  restricts  us  to  3 
phase  boron  implanted  CCD's  made  by  FORD  Aerospace.  There  are  three  sizes 
presently  made  by  FORD  --  512x512,  1024x1024,  and  2048x2048.  The  two  sizes 
applicable  here  are  the  512  and  the  1024  chips. 

The  512x512  chip  is  a  Photometries  propriety  chip  and  using  this  chip  restricts 
us  to  using  Photometries  hardware  and  camera  heads.  This  is  not  necessarily  a 
drawback  as  Photometries  makes  good  products.  The  1024x1024  chip  is  made  for 
S.A.I.C.,  but  FORD  will  run  lots  for  other  customers.  It  may  be  possible  to  attain  a  1024 
chip  through  Lockheed  as  they  have  ordered  a  lot.  There  are  some  advantages  and 
drawbacks  to  both  chips  that  will  be  discussed  in  detail. 


fa)  CCD  Physical  Characteristics 

Using  these  CCD's  in  MPP  mode  reduces  dark  noise  by  a  factor  of  30,  thus 
substantially  increasing  integration  times  allowed  on  the  CCD  (which  increases 
sensitivity  of  the  camera)  and/or  decreases  the  cooling  requirements  of  the  camera 
head  thus  allowing  for  a  smaller  head  design.  It  is  these  characteristic  of  MPP 
operation  that  makes  these  CCDs  ideal  for  us. 

The  FORD  CCD  masks  have  a  pixel  size  of  18um,  thus  the  physical  size  of  each 
CCD  image  area  is: 

512x512:  512x18um  =  9.2mm 

1024x1024:  1024  x  18um  =  18.4mm 

[The  Tektronics  chip  we  used  in  the  ASIPII  system  (TEK512)  had  a  pixel  size  of  27um, 
and  thus  an  image  area  of: 

512x512:  51 2  x  27um  =  13.7mm  ] 

The  physical  size  is  an  important  parameter  to  consider  as  it  determines  what  kind  of 
optical  demagnification  is  necessary  to  image  the  intensifier  (24mm)  down  to  the  CCD. 
This  coupling  ratio  influences  the  efficiency  of  the  coupling  optics  (because  of  the 
limited  choice  of  suitable  lenses)  and  thus  the  sensitivity  of  the  CCD  to  the  image 
intensifier  image.  The  importance  of  this  efficiency  will  be  discussed  in  a  later  section. 


To  couple  the  CCD  to  the  intensifier,  commercially  available  relay  lenses  (a 
collimator  and  camera  lens  in  tandem,  with  focal  lengths  f  and  f  ,  and  F  numbers  F 
and  F  respectively)  will  be  used.  The  efficiency  of  coupling  is  given  by: 


(1/(1  +4*F  r*2)) 

if  fiFi 

f2F2 

or 

(1/(1  +4*F1*‘2))‘((fiF  1)/(f2F2))**2 

if  fiFi 

f2F2 

Coupling  ratios  for  the  CCDs  are: 

TEK512:  13.7mm/24mm  =0.57 

PM512:  9.2mm/24mm  =0.38 

1024:  18.4mm/24mm  =0.77 


In  the  ASIPII,  we  used  a  100mm  FI  .4  collimating  lens  coupled  to  a  50mm  F0.7  lens 
which  imaged  the  intensifier  down  onto  the  CCD.  Because  we  are  limiting  ourselves 
to  commercially  available  lenses  with  good  vignetting  characteristics,  efficiencies  are 
determined  by  lens  speeds  that  we  can  buy  off  the  shelf.  For  the  512  and  1024  MPP 
chips  we  would  use: 


512x512: 

1024x1024: 


100mm  FI  .4 
75mm  F1.1 


42mm  F0.7 
50mm  F0.7 


(Note  in  the  first  case  the  CCD  lens  limits  efficiency,  whereas  in  the  second 
case  the  collimating  lens  limits  efficiency.) 

Thus,  in  coupling  to  the  larger  CCD  we  gain  a  factor  of  two  in  efficiency.  (This  is  not 
because  of  any  optical  reason,  but  results  from  limiting  lens  selection  to  commercially 
available  lenses  designed  for  relay  application.) 

The  other  important  physical  characteristic  to  consider  between  the  two  CCD's 
is  the  full  well  depths.  The  18um  pixel  has  a  normal  well  depth  of  250Kelectrons  (Ke), 
but  when  used  in  the  MPP  mode  the  well  depth  decreases  by  a  factor  of  two.  Thus,  in 
MPP  mode,  a  pixel’s  full  well  capacity  is  only  125Ke.  Compared  to  the  700Ke  of  the 
TEK512  chip  which  was  chosen  for  large  dynamic  range,  this  is  very  small.  The 
horizontal  pixel  size  (where  each  pixel  is  shifted  into  during  a  read)  is  twice  the  normal 
size,  and  the  final  summation  well  is  4  times  as  big.  In  addition,  these  areas  on  the 
CCD  are  not  operated  in  MPP  mode  as  they  are  cleared  directly  before  reading  and 
thus  dark  noise  is  not  a  problem. 

The  full  well  capacity  of  the  horizontal  pixels  is  500Ke,  and  the  summation 
amplifier  is  lOOOKe.  Thus,  the  dynamic  range  of  the  chip  can  be  greatly  increased 
using  binning.  The  greatest  amount  of  dynamic  range  available  to  the  chip  would  be 
adding  9  pixels  together  into  the  summation  well  giving  lOOOKe.  This  requires  3x3 
binning  and  3  pixels  would  be  added  to  each  horizontal  well  (3  x  125Ke  =  375Ke) 
which  is  within  the  capacity  of  a  horizontal  well.  2x2  binning  would  give  us  a 
summation  well  of  500Ke.  These  results  are  summarized  below: 


512x512  Chip; 


Binning 
1x1  binning 
2x2  binning 
3x3  binning 


Resolution 

512x512 

256x256 

170x170 


Full  Well  Capacity 
1 25Ke 
500Ke 
lOOOKe 


1024x1024  Chio: 


_ Binning  Resolution 

1x1  binning:  1024x1024 

2x2  binning:  512x512 

3x3  binning:  340x340 


Full  Well  Capacity 
125Ke 
500Ke 
lOOOKe 


We  can  see  that  both  chips  are  capable  of  the  same  dynamic  range  except  that  if  we 
want  to  set  up  a  system  to  utilize  the  maximum  dynamic  range  of  the  CCD,  the  1024 
chip  would  have  twice  the  resolution.  In  practice,  we  have  found  that  3x3  binning  on  a 
512x512  chip  is  useful  for  storing  data  in  minimum  space,  but  the  spatial  resolution  is 
not  very  good.  512x512  images  are  very  beautiful  but  take  a  long  time  to  read/write 
and  take  up  a  great  deal  of  data  space.  We  have  been  using  256x256  images  in  the 
ASIPII,  which  have  been  a  good  compromise  for  dynamic  range,  resolution,  speed  of 
image  aquisition  and  display,  and  data  storage  capabilities. 

The  1024  chip's  physical  characteristics  give  it  an  advantage  over  the  512  chip 
in  that  the  coupling  from  the  intensifier  is  more  efficient  and  that  for  the  same  dynamic 
range  (lOOOKe),  the  resolution  of  the  chip  would  be  340x340  pixels  which  might  be  a 
nice  compromise  between  the  256  and  the  512  modes.  These  differences  also  have 
to  be  looked  at  in  terms  of  read  time  and  data  storage.  Briefly,  a  typical  read  rate 
would  be  500KHz  and  a  typical  optical  disk  system  would  be  650Mb  per  disk  (with 
each  16  bit  pixel  value  requiring  2  bytes  of  8  bit  storage): 


Resolution 

1024x1024 

512x512 

340x340 

256x256 


Read  Time 
2.0  sec 
0.5  sec 
0.2  sec 
0.13  sec 


Imaaes/Disk 
325  images 
1300  images 
3250  images 
5200  images 


Time  (@  4  im/minl 
1  hr  21  min 
5  hr  25  min 
13  hr  32  min 
21  hr  40  min 


The  time  parameter  is  an  indication  of  how  long  an  optical  disk  would  take  to  be  filled 
with  a  normal  data  aquisition  rate  of  4  images/minute.  Since  it  takes  about  .25  to  .5 
seconds  for  the  filter  to  change  to  its  next  position,  either  the  340x340  (1024x1024  at 
3x3  binning)  or  the  256x256  (512x512  at  2x2  binning)  options  are  reasonable 
operating  parameters  for  us  to  use.  The  1024x1024  chip  would  double  our  full  well 
capacity  and  give  us  better  picture  resolution  than  the  512x512  chip,  but  the  512x512 
chip  would  enable  us  to  store  1 .77x  more  data  on  a  disk. 


(b)  CCD  Resolution  vs.  Intensifier  Resolution 


When  deciding  between  resolutions  of  256x256  and  340x340,  it  is  important  to 
see  if  there  is  any  gain  between  the  two  in  terms  of  system  resolution.  The  intensifier 
has  a  resolution  of  15  Ip/mm  at  a  Modulation  Transfer  Function  of  25%,  which  for  a 
24mm  image  gives  us  360  line  pairs.  Thus,  if  the  CCD  columns  are  lined  up  perfectly' 
with  the  resolution  pattern  lines,  a  minimum  of  720  pixels  would  be  needed  to  get  this 
maximum  resolution  thus  a  512x512  image  probably  matches  fairly  well  with 
intensifier  resolution.  At  higher  binning  (lower  resolution),  the  resolution  is  limited  by 
the  CCD,  so  resolution  of  the  340x340  image  is  better  than  the  256x256  image  by  a 
factor  of  340/256  or  1.33.  At  an  MTF  of  60%,  the  image  intensifier  resolution  is  only  7.5 
Ip/mm  which  gives  180  line  pairs,  matching  closely  with  a  340x340  image. 

From  experience,  256x256  is  adequate  resolution  for  auroral  work  to  resolve 
the  spatial  phenomenon  being  observed.  Any  upgrade  of  resolution  increases  the 
quality  of  the  image,  but  it  should  be  noted  there  is  probably  little  real  increase  in 
scientific  information  for  auroral  phenomenon,  so  both  formats  would  give  adequate 
resolution. 


(c)  Dynamic  Range  and  Read  Electronics 

We  previously  discussed  utilizing  the  full  dynamic  range  of  the  CCD  by 
optimizing  the  binning.  Now  we  consider  converting  the  electrons  in  the  summation 
well  to  Analogue  Digital  Units  (ADU's)  to  read  into  the  computer  memory.  To  achieve 
fast  read  rates,  12  bits  of  digitization  are  typically  used.  The  higher  the  resolution,  the 
lower  the  read  rates.  Read  rate  also  determines  to  large  extent  the  read  noise 
associated  with  the  conversion. 

At  500KHz,  one  can  expect  a  read  noise  of  around  50  electrons.  If  we  set  the 
minimum  ADU  to  be  twice  this  read  noise,  we  get  an  ADU  of  100  electrons.  With  12  bit 
resolution,  our  dynamic  range  will  be  limited  to  4000:1.  Thus,  if  one  ADU  is  100 
electrons,  the  A/D  converter  will  saturate  at  400Ke  regardless  of  when  the  CCD  will 
saturate.  This  matches  well  with  a  full  well  capacity  of  500Ke  (i.e.  the  PM512  at  2x2 
binning). 

To  achieve  a  full  well  capacity  of  lOOOKe  (i.e.  the  1024  chip  at  3x3  binning)  with 
a  12  bit  A/D  converter,  one  would  have  to  set  1  ADU  to  equal  250  electrons.  The 
dynamic  range  would  still  be  4000:1  though,  so  as  far  as  the  data  is  concerned,  there 
is  no  increase  in  dynamic  range  of  the  system.  Another  gain  could  be  included  in  the 
electronics  (G=x2.5)  to  set  1  ADU  =  100  electrons  if  higher  sensitivity  was  also 
needed. 

If  one  was  to  use  the  1024  chip  at  3x3  binning  and  fully  utilize  the  dynamic 
range  of  the  CCD  (ratio  of  read  noise  to  full  well  depth),  one  would  have  to  use  a  14  bit 
A/D  converter  setting  the  maximum  dynamic  range  at  16,000:1.  1  ADU  would  be 
equal  to  60  electrons  and  the  A/D  converter  would  saturate  at  lOOOKe. 


Summary  of  Dynamic  Range 


Full  Well  Capacity 

Bits 

Dynamic  Range 

ADU  (min) 

Ims*) 

500Ke 

12 

4000:1 

lOOe 

400Ke 

1 OOOKe 

12 

4000:1 

250e 

lOOOKe 

lOOOKe 

14 

16000:1 

60e 

1 OOOKe 

In  physical  terms,  a  12  bit  dynamic  range  is  probably  fully  adequate  for  an 
instrument  for  measuring  auroral  and  airglow.  For  instance,  on  one  sensitivity  setting, 
one  could  look  at  airglow/aurora  ranging  from  25  Rayleighs  to  100,000  Rayleighs. 
Dynamic  range  is  also  determined  by  the  image  intensifier  gain  which  determines  the 
size  of  a  photon  event  at  the  CCD.  Before  deciding  which  CCD  is  better  for  us,  it  is 
necessary  to  discuss  this  aspect  of  the  design. 


(d)  Photon  Limited  Operation 

The  first  thing  to  look  at  in  the  system  is  to  see  whether  each  photon  detected  at 
the  image  intensifier  gives  us  a  detectable  signal  at  the  CCD.  Intensifier  gain  is 
quoted  from  VARO  as  50,000,  however  corrected  for  green  light  this  reduces  to  about 
10,000.  This  gives  us  the  average  gain  for  a  stream  of  photons  coming  into  the 
intensifier  versus  the  output  signal.  Since  the  quantum  efficiency  for  the  cathode  is  of 
the  order  of  10%,  the  gain  for  an  individual  photon  that  gets  detected  by  the  cathode 
must  be  on  the  order  of  100,000. 

Assuming  photon  statistics,  we  define  a  usable  picture  as  one  with  a  signal  to 
noise  ratio  of  2:1,  thus  a  minimum  detectable  picture  is  4  photoelectron  events.  This 
would  give  us  400,000  electrons  out  of  the  intensifier.  For  a  relay  optics  of  10% 
efficiency  (i.e.  PM512),  40,000  photons  would  reach  the  CCD  spread  over  an  average 
of  four  pixels  with  a  quantum  efficiency  of  30%.  Thus,  one  detectable  event  at  the 
image  intensifier  gives  us  nearly  3000  electrons/pixel  in  the  CCD.  For  the  relay  lens 
coupling  for  a  1024x1024  chip,  this  number  would  be  around  5000  electrons/pixel. 

This  points  to  a  problem  in  dynamic  range.  If  each  event  is  on  the  order  of  3000 
electrons,  and  our  maximum  full  well  is  500Ke,  then  we  only  have  a  dynamic  range  of 
166:1  (8  bits)  for  a  512x512  chip,  and  333:1  (9)  bits  for  a  1024x1024  chip.  This  means 
that  read  noise  and  dark  current  noise  sources  are  negligible  and  that  the  image 
intensifier  gain  is  restricting  our  dynamic  range.  Using  the  lowest  gain  on  the  image 
intensifier  reduces  gain  by  a  factor  of  10,  thus  a  minimum  event  would  be  on  the  order 
of  300  to  500  electrons.  This  enables  us  to  have  a  dynamic  range  of  only  1 1  bits. 

It  may  be  that  we  will  have  to  decrease  the  coupling  efficiency  of  the  relay  optics 
by  putting  in  an  aperture  or  by  using  relay  lenses  with  smaller  apertures  (which  could 
decrease  the  size  and  cost  of  the  system).  It  seems  that  ideally  we  would  want  to 
design  a  system  with  a  minimum  detectable  event  at  2-4  times  the  read  noise/dark 
noise.  With  a  read  noise  of  50  electrons,  a  good  minimal  detectable  event  would  be 
100  to  200  electrons.  This  would  give  us  a  dynamic  range  on  the  order  of  12  bits. 


It  should  be  noted  that  there  is  no  disadvantage  of  using  the  interim  <,!  at  a  low 
gain,  in  fact  it  decreases  the  intensifier  noise  and  thus  gives  a  longe  i.vegraiion 
capability.  The  dynamic  range  of  the  intensifier  is  determined  by  how  ...  light  is 
entering  the  photocathode.  Reducing  gain  at  the  intensifier  would  enable  us  to 
increase  the  dynamic  range  at  the  camera  for  very  bright  aurora  by  decreasing  the 
number  of  electrons  in  the  CCD  per  photon  event  at  the  cathode.  Let's  take  a  specific 
design  example: 

Relay  lens  coupling  at  F4.0:  Efficiency  is  0.01 5 

4  photoelectrons  gives: 

450  electrons/pixel  at  CCD  for  Max  gain 
225  electrons/pixel  at  CCD  for  Max/2 
45  electrons/pixel  at  CCD  for  Max'iO 

and  we  require  2x  read  noise,  i.e.  100  electrons/pixel  for  a  detectable  event. 

Hence: 


CCD  Size 

Int.  Gain 

Min.  event 

Ma>-  e-'-n? 

512 

Max 

4pe  --  450e 

1 000pe--bu0.'.t- 

512 

Max/2 

4pe  -  225e 

2000pe  -500Ke 

512 

Max/1 0 

lOpe  --100e 

1 0OO0pe--500Ke 

1024 

Max 

4pe  -  450e 

2000pe--lMe 

1024 

Max/2 

4pe  -  225e 

.  4000pe-1Me 

1024 

Max/10 

lOpe  -lOOe 

20000pe--l  Me 

Here,  a  512x512  CCD  is  assumed  to  be  2x2  binned  giving  a  dynamic  range  of 
500Ke,  and  a  1024x1024  CCD  is  3x3  binned  giving  it  a  dynamic  range  of  1  Million 
electrons  (Me).  For  instance,  using  a  512x512  CCD  with  the  intensifier  on  Maximum/4 
in  2x2  binning  mode,  would  give  us  a  minimum  photoelectron  event  of  100  electrons 
and  a  maximum  of  500Ke  with  a  dynamic  range  of  about  5000:1  matching  a  12  bit 
ADC  fairly  well.  Using  a  1024x1024  chip  in  3x3  binning  mode,  we  could  use  the 
intensifier  at  Max/2  giving  a  minimum  event  225  electrons  and  a  dynamic  range  of 
4000:1. 


(e)  System  Sensitivity 

The  final  thing  to  check  is  whether  the  system  sensitivity  is  affected  by  choosing 
either  the  1024x1024  or  the  512x512  CCDs.  If  we  require  a  usable  image  as  in  the 
last  section  to  be  4  photoelectrons,  then  we  need  at  least  4  photoelectrons, pixel.  This 
gives  us  our  minimum  sensitivity  since  we  have  already  shown  that  for  each  4 
photoelectron  event  the  CCD  generates  more  than  enough  electrons  to  exceed  CCD 
noise  sources.  For  a  1  Rayleigh  source,  filter  transmission  0.6,  image  cone  at 
intensifier  F2.0,  Q.E.  of  intensifier  of  10%,  and  51 2x51 2  resolution  elements,  it  can  be 
shown  that  there  are  0.02  photoelectron/pixel  from  the  image  intensifier  cathode. 
Thus,  for  a  512x512  image,  a  minimum  image  of  4  photoelectron/pixel  requires  200  R 
sec.  A  1024x1024  image  requires  800  R  sec. 


Thus: 


PM512  2x2  binned  256x256:  50  R  sec 
1024  3x3  binned .  340x340:  80  R  sec 


Thus,  a  1024x1024  chip  operated  in  3x3  binning  is  less  sensitive  than  the  512x512 
chip  operated  in  2x2  binning. 


(f)  Conclusion 

Thus  the  512  supplied  by  Photometries  is  the  CCD  that  is  best  for  our 
application.  At  256x256  resolution,  it  offers  the  highest  sensitivity,  and  is  comparable 
in  most  parameters  at  4000:1  dynamic  range.  It  is  also  less  expensive  than  the  1024 
and  supplied  by  Photometries  with  whom  we  have  a  good  working  relationship. 

There  are  two  reasons  for  using  the  1024x1024  chip.  The  first  is  if  we  wanted  to 
achieve  a  higher  dynamic  range  on  the  order  of  16000:1.  The  second  is  that  we  could 
increase  the  noise  headroom  of  the  system  by  using  this  chip  at  340x340  mode,  and 
defining  the  least  significant  event  as  250  electrons  and  matching  the  relay  optics  to 
this.  This  would  give  a  higher  resolution  image  more  immune  to  system  noise  than  the 
512x512  chip  with  the  same  dynamic  range.  It  would  also  take  longer  to  read,  write 
and  display  images  and  take  up  more  data  storage  space.  Our  experience  has  been 
that  the  higher  resolution  is  not  really  quantitatively  significant,  but  rather  just  makes 
the  image  prettier  and  that  system  noise  is  not  a  problem  (especially  with  the  new 
cathode  cooler  for  the  image  intensifier).  The  main  'noise'  source  in  practice  is  the  sky 
background  continuum. 
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Image  Processing  Overview  for  MIP  Camera  Keo  Consultants  7/7/9C 

The  following  description  gives  an  overview  of  the  Image  Processing  needs  for 
the  MIP  camera  as  presently  conceived.  These  needs  may  be  thought  of  as  a  basic  set 
of  operations  that  we  wish  to  be  able  to  implement  quickly  and  easily  from  our 
hardware/software  environment.  As  the  camera  is  used  in  a  research  environment,  we 
want  to  have  a  system  that  can  be  expanded  and  improved  with  minimum  effort. 

(a)  MIP  Data  Set  and  Acquisition  Cycle 

The  MIP  will  be  used  in  long  term  observations.  A  typical  data  set  will  contain 
four  images  and  repeat  every  30  seconds  to  a  minute  depending  on  the  phenomenon 
being  observed.  Sometimes  the  data  set  will  be  only  one  image,  and  sometimes  there 
may  be  up  to  5  images  being  recorded  at  once.  However,  we  consider  the  normal  set 
to  be  four  images  per  cycle. 

The  camera  is  a  slow  scan  camera  with  a  512x512  CCD  clocked  at  between 
500KHz  to  1.1MHz.  The  final  clock  rate  is  to  be  determined  by  the  read  noise  of  the 
summing  amplifiers.  An  exposure  can  be  anywhere  from  1/4  second  to  20  seconds, 
but  typically  this  will  be  about  one  to  four  seconds.  At  full  resolution,  the  read  time  of 
the  CCD  will  be  about  1/4  of  a  second.  However,  typical  resolution  will  be  2x2  binned, 
or  256x256,  thus,  the  read  time  of  the  CCD  will  be  about  1/1 6th  of  a  second. 

The  CCD  output  will  have  12  bit  resolution. 

(b)  KEO  Image  Processing  Environment 

There  are  two  major  options  for  image  processing  environments.  The  first  is  the 
Macintosh  llci  using  a  custom  designed  board  to  interface  the  camera  to  the  Macintosh 
and  using  commercial/custom  software  to  control  the  camera  and  images.  The 
drawbacks  are  that  both  the  hardware  components  (the  Mac  and  the  interface  board) 
are  costly,  and  that  all  image  processing  has  to  be  done  in  the  computer’s  CPU. 
Accelerator  boards  could  be  added  to  speed  up  processing,  but  this  increases  costs.  In 
addition,  considerable  programming  effort  would  be  needed. 

The  second  option  is  to  use  an  IBM  clone  platform  running  a  powerful  386  at 
25MHz  with  4  Mbytes  of  RAM,  and  couple  this  with  commercially  available  image 
processing  hardware  (Figure  1).  Any  IBM  solution  requires  the  capability  of  migrating 
from  16  bit  operating  systems  (MS-DOS/Windows  3.0)  to  a  true  32  bit  OS. 
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Figure  A :  KEO  System  Overview 

Image  Processing  hardware  availability  is  mainly  restricted  by  the  need  to  have 
a  12  bit  digital  input  port.  There  are  three  companies  that  presently  make  board  sets 
with  this  function  available:  MATROX,  Image  Technology  Inc.,  and  CORECO  (although 
their  product  is  not  yet  to  market).  This  report  attempts  to  define  our  needs  in  image 
processing  in  order  to  fully  investigate  which  environment  will  be  most  suited  to  this 
application. 

Because  this  instrument  is  being  designed  as  a  small  field  instrument,  we  have 
restricted  the  field  configuration  to  have  only  one  monitor.  Another  monitor  could  be 
added  back  in  the  lab  to  facilitate  data  analysis.  It  would  be  preferable  to  have  a 
monitor  that  fits  in  a  19  inch  rack,  so  resolution  might  be  a  factor  (i.e.  are  there 
1280x1024  monitors  that  are  small  enough  to  be  rack  mounted?). 

This  design  aspect  requires  the  ability  to  have  either  some  sort  of  VGA  mixing 
scheme  (i.e.  Image  Technology),  or  VGA  switching  (MATROX).  A  quick,  efficient  user 
interface  to  control  all  the  controls  of  the  camera  will  have  to  be  developed.  A  final 
analysis  of  how  our  design  could  be  implemented  on  each  of  the  above  systems  will  be 
the  subject  of  another  report  (see  Appendix  3). 
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As  stated  above,  the  images  will  normally  be  acquired  as  256x256  images 
clocked  in  at  1.1  MHz.  Sometimes  high  resolution  images  will  be  desired  at  512x512 
resolution  (Figure  2). 
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Figure  2:  Image  Format 

Table  1  shows  the  capabilities  of  both  the  MATROX  and  the  ITI  IP  board  sets: 

Company  Memory  Frame-Formal  256x256.  512x512 

MATROX  4  Mbytes  2Kx1Kx16  bits  32  8 

ITI,  Inc.  2  Mbytes  1Kx1Kx16bits  16  4 

Table  1 :  Number  of  Images  stored  in  a  Frame  Buffer 

Each  data  set  can  be  considered  as  one  block  of  memory  to  further  divide  up  the  total 
frame  buffer.  For  instance,  to  view  a  data  set  of  four  images  (the  normal  mode  of 
operation),  we  would  have  four  images  contiguous  in  the  frame  buffer  and  display  this 
section  of  memory.  (Figure  3)  For  normal  resolution,  this  would  gives  us  a  512x512 
area  of  the  frame  buffer  and  for  high  resolution,  this  will  give  us  a  1024x1024  area. 
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For  each  image  we  will  discuss  the  typical  image  processing  needs  that  might 
be  implemented  during  data  acquisition.  Keep  in  mind  that  these  operations  have  to 
be  carried  out  for  each  image  in  the  data  set.  (Figure  4) 

The  raw  data  would  be  acquired  and  stored  in  the  Raw  data  area.  Subsequent 
operations  would  be  subtracting  a  noise  image,  multiplying  the  image  by  a  flat-field 
correction  (different  for  the  four  images  because  of  the  different  filters  used),  taking  a 
histogram  of  the  data  (not  necessarily  every  cycle),  and  rescaling  the  image  based  on 
the  results,  and  finally  outputting  this  result  to  the  display  area  so  that  four  images  can 
be  displayed  on  the  screen,  each  maximized  for  contrast  based  on  its  own 
characteristics.  (  See  Figures  6,7,8.) 

Using  image  processing  hardware  with  a  set  Frame  Buffer,  one  would  then 
divide  the  buffer  into  four  regions  so  that  the  data  could  be  quickly  used  for  analysis 
(Figure  5).  [Background  and  Flat-Field  images  would  be  read  in  before  the  data 
acquisition  process.] 


Figure  4:  Data  Process  and  Associated  Frames  Needed 
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Figure  6:  Mathematical  Corrections  for  Data 
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Figure  5:  Representation  of  Frame  Buffer 
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Figure  7:  Create  Display  Image  based  on  image  statistics 
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(a)  12  bits  Mapped  to  8  bits  Monochrome  Output 
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(b)  12  bit  Psuedo-Color  Map  to  Color  Output 


Figure  8:  Output  Display  Image  in  either  Monochrome  or  Psuedo-Color 

Most  of  these  operations  do  not  require  real-time  computations.  In  reality,  if  we 
collect  four  images  every  minute  with  a  2  second  exposure.  We  will  have  typically  40 
seconds  of  processing  time  for  each  data  set.  Most  of  the  aspects  of  the  image 
processing  system  that  concern  us  in  this  process  are  the  fact  that  the  image  is  12  bits 
deep  rather  than  the  typical  8  bits,  and  how  the  image  processing  hardware  handles 
this.  In  software,  this  is  not  an  issue  (i.e.  on  the  MAC)  but  on  the  image  processing 
boards  where  you  are  limited  to  a  finite  number  of  functional  operations,  this  has  to  be 
considered. 

Two  areas  of  concern  are  flat-fielding,  and  display.  Flat-fielding  is  used  to 
correct  for  known  optical  effects  and  CCD  inconsistencies  that  are  functions  of  the 
camera.  Such  corrections  would  certainly  never  approach  a  factor  of  16,  and  expected 
maxima  are  3  -5.  To  implement  a  high  resolution  flat-field,  one  needs  to  multiply  the 
data  by  a  correction  factor  and  then  subsequently  normalize  the  pixel  field.  An 
example  for  16  bit  correction  normalization  to  16  would  be: 

(i)  Pixel  x  16  bit  flat-field  Pixel 

(ii)  Result  =  Result/16 
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If  you  wanted  a  flat-field  of  3.5,  the  16  bit  flat-field  Pixel  would  be: 

3.5  x  (2**1 6)/1 6 

Normalizing  to  16  gives  us  the  advantage  that  to  normalize  the  result  requires 
only  an  Arithmetic  Shift,  rather  than  a  divide.  Another  factor  in  using  16  is  that  final 
result  from  a  12  bit  image  will  always  fit  inside  a  16  bit  data  buffer. 

The  problem  is  that  a  16  bit  by  12  bit  multiplication  requires  28  bits  for  storing  the 
result,  and  most  boards  do  not  have  appropriate  hardware  capability.  An  alternate 
scheme  has  been  developed  using  multiple  8  bit  divides  to  get  around  this  problem 
and  is  discussed  in  Appendix  4. 

Displaying  the  results  of  a  data  set  requires  some  thought  only  in  that  the  output 
DAC's  are  set  up  for  8  bit  resolution,  and  the  data  is  typically  12  bit.  Data  could  be 
scaled  down  so  that  the  display  buffer  is  only  8  bits  deep,  or  perhaps  the  output 
hardware  supports  a  12  in,  8  out  LUT.  This  would  be  ideal  for  psuedo-color 
implementation,  where  we  could  define  a  color  spectrum  that  spans  a  full  4096  color 
range  (Figure  9). 

This  describes  the  basic  "automatic"  mode  of  the  MIP  camera  in  terms  of  image 
processing  needs.  To  summarize:  Each  data  cycle  contains  four  images,  and  repeats 
every  30  -  60  seconds.  For  each  image  collected: 

(i)  Acquire  image  into  Raw  data  buffer 

(ii)  Subtract  Noise  from  Noise  buffer 

(iii)  Do  a  flat-field  with  data  from  Flat-field  buffer 

(iv)  Every  Nth  image,  do  a  histogram 

(v)  Create  LUT  based  on  histogram  results 

(vi)  Create  a  transform  image  in  display  buffer  through  LUT 

(vii)  Set  display  characteristics  and  display  this  buffer 

Neither  the  MATROX  board,  nor  the  ITI  board  would  be  taxed  by  this  application  in 
terms  of  execution  speed.  In  fact,  the  boards  would  be  greatly  under-utilized.  However 
having  the  powerful  capability  of  these  boards  would  allow  us  to  develop  a  field 
instrument  with  the  power  of  some  of  the  processing  computers  used  in  the  lab,  and 
would  allow  the  scientist  in  the  field  many  more  options  than  an  instrument  purely 
targeted  for  data  acquisition.  Images  could  be  analyzed  during  the  off  periods 
(daytime,  bad  weather,  etc.).  Tasks  could  be  multi-tasked  (since  the  IP  hardware  takes 
care  of  most  of  the  work),  and  one  could  envision  updating  an”observation  notes"  file 
while  collecting  data. 
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Figure  9:  12  bit  in,  8  out  Psuedo-color 


In  the  largest  view,  perhaps  some  of  the  processing  that  was  before  restricted  to 
non-"real  time"  applications  could  be  implemented.  A  few  operations  will  be  discussed 
in  brief  in  the  next  Section. 


(d)  Future  Applications 


The  easiest  applications  that  one  envisions  implementing  are  graphics  and 
representation  of  data  from  other  instruments  in  spatial  coordinates  relating  to  the  MIP 
data.  Such  applications  include  representing  radar  data,  position  of  magnetic 
coordinates,  position  of  the  sun  with  respect  to  the  auroral  oval,  etc. 


Magnetic  Coordinates  Radar  Data 

Figure  10:  Example  Graphics  Applications 

Another  application  to  implement  on  the  IP  hardware  could  be  the  geometrical 
transformations  that  change  the  fish-eye  coordinate  system  of  the  MIP  to  a  real 
longitude/latitude  coordinate  system  (Northwest  Transformation).  This  is  currently 
being  done  on  a  286  system  and  takes  an  unacceptable  time.  Implementing  this 
algorithm  on  the  MIP  would  make  this  instrument  a  very  powerful  lab  analysis  tool  as 
well  as  the  possibility  of  having  a  fast  enough  system,  that  this  analysis  could  be  done 
in  real  time. 

There  is  a  question  as  to  whether  this  transformation  can  be  implemented  with 
the  finite  number  of  hardware  tools  provided  either  by  MATROX  or  by  ITI.  Further 
research  needs  to  be  done  to  see  whether  this  will  have  any  effect  on  the  Image 
Processing  hardware  choice. 


Figure  1 1 :  Representation  of  Northwest  Transformation 

Finally,  if  we  add  a  Fabry-Perot  to  the  front  end  of  the  MIP,  it  would  create  a 
camera  that  requires  a  great  deal  of  processing  power  to  acquire  data.  The  basic 
scheme  would  be  to  take  10  images  (256x256)  in  succession  each  with  a  different 
Fabry-Perot  spacing.  For  each  pixel,  there  would  be  a  10  point  curve  which  would  be 
Least  Square  Fitted  and  a  maximum  determined  for  the  curve  which  corresponds  to  a 
velocity.  This  velocity  would  then  be  displayed  on  an  overlay  of  an  image  as  vectors, 
much  the  same  way  the  radar  data  is  overlayed  on  the  data. 

These  are  just  of  few  of  the  more  complicated  applications  that  one  may  want  to 
implement  on  the  new  MIP.  Obviously  many  more  will  be  identified  as  the  instrument  is 
used.  Thus,  in  conceiving  of  a  final  design,  we  plan  to  have  a  hardware/software 
system  that  matches  our  present  needs  well  and  provides  a  good  path  for  future 
development. 
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Imaging  Processing  Analysis  Keo  Consultants  6/6/90 

1.  Macintosh  Solution; 

The  Macllci  is  a  very  powerful  computer  housed  in  a  small  package.  From  this 
point  of  view  it  makes  a  very  convenient  field  package,  especially  because  of  the 
graphical  user  features.  Images  and  menus  would  all  be  displayed  on  the  same 
screen  which  has  a  resolution  of  640x480.  This  could  be  upgraded  with  additional 
hardware  to  include  larger,  higher  resolution  screens,  multiple  screen  applications, 
and  accelerator  boards  for  graphics  operations.  In  practice  though,  this  would  not 
really  be  necessary.  One  drawback  of  using  the  Macllci  is  that  you  are  limited  to 
Macintosh  hardware.  At  present,  the  prices  are  much  higher  than  for  the  IBM  market, 
and  there  is  a  very  limited  amount  of  add-on  boards  that  are  useful  for  our  image 
processing  needs.  This  is  changing,  though,  and  many  companies  are  bringing  out 
more  products  for  the  Mac  II  series. 

Another  drawback  is  that  GL  uses  primarily  PC's,  and  so  do  most  of  the  other 
labs  working  in  the  atmospheric  community.  Thus  a  Macintosh  Instrument,  while  small 
and  powerful,  might  cause  some  transportability  of  data  problems  down  the  line. 

Using  the  NuBus200  board  for  instrument  control  has  the  advantage  of  using 
the  Photometries  hardware.  We  know  this  will  work  well  with  our  camera  head, 
because  it  has  been  specifically  designed  for  the  system.  It  has  not  yet  been  modified 
for  our  camera  though,  so  delivery  could  be  quite  long  (6  months  would  not  be 
unreasonable).  The  Photometries  software  package  is  very  well  written  for  scientific 
data  aquisition,  and  is  well  proven  in  the  old  ASIPII.  In  addition,  it  will  be  incorporated 
in  with  the  IPLab  software  package,  which  provides  a  large  library  of  menu-driven 
image  processing  functions.  Using  the  NuBus  200  Source  code  in  MPW  C++,  and  the 
IPLab  User  Customization  Code  in  MPW  C,  we  can  easily  incorporate  a  KEO  user's 
system  into  the  Macintosh  and  have  it  operating  as  any  other  Mac  application  would 
run. 


Experience  has  shown  that  Mac  applications  and  system  software  are  very 
memory  intensive,  and  thus  we  would  need  to  put  at  least  4  Mbytes  of  RAM,  but  would 
be  safer  with  8Mbytes  for  ultimate  flexibility.  All  images  would  be  stored  in  the 
computer  RAM,  and  thus  there  is  as  much  flexibility  for  display  and  manipulation  as 
any  other  data  array,  or  window.  Using  the  standard  QuickDraw  routines,  images  will 
be  able  to  be  dragged,  re-sized,  and  manipulated  as  any  other  window  on  the  screen 
Thus,  this  software  implementation  would  allow  us  maximum  flexibility  in  how  many 
images  are  displayed,  how  big  they  are,  etc.  In  addition,  the  QuickDraw  routines, 
allow  for  easy  creation  of  pallets  and  all  the  necessary  graphics  systems. 

Writing  the  software  as  a  Macintosh  application  would  be  straight  forward,  and 
would  mostly  involve  developing  on  the  existing  structure  of  the  Nu200  Software  and 
IPLab,  rather  than  writing  our  own  user  system.  There  would  be  the  initial  effort  o. 


learning  MPW  which  is  a  complicated  programming  environment,  and  in  addition,  we 
would  have  to  learn  C++  which  is  an  extensive  Object-Oriented  Language.  This  would 
provide  very  advanced  code,  but  would  take  a  great  deal  of  effort  for  another  person  to 
pick  up  and  learn. 

The  NuBus200  card  has  the  drawback  of  not  being  able  to  take  images  from 
any  other  camera  device,  unless  it  was  in  strict  accordance  with  the  polled  I/O 
Photometries  method.  Thus,  with  this  board  in  the  computer,  no  other  camera  could  be 
be  used.  In  addition,  the  processing  power  of  the  work  station  is  limited  by  the 
processing  power  of  the  computer  since  all  the  work  is  done  inside  of  the  Macllci. 
There  are  more  image  processing  accelerator  boards  and  other  hardware  peripherals 
becoming  available,  but  they  will  add  costs  to  an  already  expensive  system. 

Memory  for  the  number  of  images  desired  is  completely  flexible,  and  the  Macllci 
does  not  have  addressing  problems  associated  with  systems  running  under  DOS. 
True  multi-tasking  while  theoretically  possible  on  the  68030,  is  very  difficult  on  the  Mac 
because  of  the  operating  system.  A  psuedo-multi-tasking  system  running  under 
MultiFinder  can  be  implemented.  Software  will  require  extensive  use  and  knowledge 
of  the  intricacies  of  the  Mac  Toolbox,  rather  than  a  small  set  of  image  processing 
related  commands.  However,  this  will  be  minimized  by  the  amount  of  software  already 
written  by  Photometries,  and  by  IPLab. 

Drawbacks  are  the  incompatibility  of  the  Mac  with  IBM  systems  which  are  the 
standard  at  GL,  and  a  de-facto  standard  right  now  in  the  scientific  community. 
Hardware  upgrades  are  expensive  and  limited  by  the  Macintosh  Market.  The  plus  of 
having  the  user  friendly  system,  is  reduced  by  the  advent  of  Windows  3.0  which  is  very 
similiar  (in  some  ways  even  better)  than  the  Mac  Operating  System.  The  plus  of  the 
Mac  in  terms  of  this  is  that  the  Windows  3.0  developer's  kit  is  not  yet  available,  and 
there  will  be  a  large  development  effort  needed  to  write  all  the  menu  drivers  for  the 
KEO  system  and  the  image  processing  system,  that  has  already  been  written  for  the 
MAC  (i.e.  example  IPLab). 

A  present  software  benchmark  for  the  NuBus200  system,  is  that  it  can  read  a 
IKxlK  image,  scale  it,  and  display  it  on  the  monitor  in  about  7  seconds  with  a  Macll  ci. 
For  a  512x512  image,  this  translates  to  2  seconds,  and  for  a  256x256:  1/2  second. 
This  is  fast  enough  for  our  applications  unless  we  desired  to  do  real-time 
adding/subtracting,  flatfielding,  etc. 


2.  IBM  Solution: 

With  today's  computer  technology  developing  so  rapidly,  it  makes  sense  to  buy 
a  computer  with  state-of-the-art  power  now,  so  that  it  will  not  become  obsolete  over  the 
next  few  years.  Unlike  the  ASIPII,  we  envision  that  the  new  MIP  will  not  only  serve  as 
a  powerful  field  instrument  but  be  a  compact,  powerful  workstation  back  in  the  lab  for 
processing  the  data.  Thus,  we  would  buy  a  386  based  machine  running  at  25MHz 
with  4  Mbytes  of  memory  and  a  large  disk  drive  (65-110  Mbytes).  These  are  very 
affordable  right  now,  and  are  available  with  high  resolution  graphics  display  drivers 


and  monitors  preparing  us  for  a  true  32-bit  workstation  that  can  be  migrated  up  into  32 
bit  operating  systems  such  as  OS/2  as  it  becomes  more  established. 

An  example  computer  would  be  the  GATEWAYS  package: 

25MHz  386  IBM  Compatible  with: 

4  Mbytes  RAM 

1.2M  5.25"  Drive 

1.44M  3.5"  Drive 

1 10M  ESDI  Drive 

ESDI  Cache  Controller 

16  Bit  VGA  with  51 2K 

14"  1024  x  768  Color  Monitor 

1  Parallel  and  2  Serial  Ports 

101  Key  Keyboard 

MS  DOS  4.01 

MS  Windows  3.0 

Floating  Point  Coprocessor 

Software: 

Microsoft  C 
Microsoft  SDK  3.0 

There  are  quite  a  few  advantages  with  going  with  the  IBM  environment.  The 
major  advantage  is  that  it  is  an  open  architecture,  and  there  are  a  very  large  number  of 
products  available  for  any  IBM  compatible.  This  gives  us  the  greatest  flexibility  in  what 
the  computer  will  configure,  what  hardware  we  want  to  use,  and  how  we  would  want  to 
upgrade  the  system  (i.e.  smaller  configurations,  ruggedized  systems,  portables,  more 
powerful  platforms  such  as  the  i486,  etc  ).  And  the  tota.  cost  is  about  half  of  a 
comparable  MAC  llci  system. 

Another  distinct  advantage  is  that  GL  currently  is  only  allowed  to  procure  IBM 
compatibles,  and  the  Macintosh  is  not  supported  through  the  Air  Force.  Therefore, 
almost  all  hardware/software  environments  already  existing  at  the  lab  are  IBM  related 
systems.  This  means  that  another  system  designed  around  the  IBM  would  remain 
compatible  with  existing  systems,  where  there  is  already  a  large  amount  of  expertise  at 
the  lab. 

Another  important  advantage  of  the  IBM  is  that  the  present  Northwest  system 
resides  on  a  286  at  the  lab.  This  would  be  easy  to  port  over  to  a  new  IBM  based 
system  whereas  as  Dave  Lucas  (who  wrote  the  Northwest  software)  feels  that  it  would 
be  totally  impractical  to  port  the  system  over  to  the  Macintosh.  The  IBM  system 
hardware  is  generally  the  least  expensive  choice  and  the  most  innovative  as  the 
market  is  so  large.  An  example  is  that  there  are  no  applicable  Image  Processing 
boards  available  to  use  on  the  Macintosh  as  there  are  on  the  IBM  series  (three 
different  companies  to  date).  This  results  in  available  hardware  solutions,  data 
portability,  and  less  new  systems  overhead. 


In  addition,  the  IBM  is  pretty  standard  in  the  research  community  (Steve  Mende, 
et.  al.  )  The  IBM  provides  us  with  an  inexpensive,  very  powerful,  and  flexible 
computing  platform  that  will  provide  us  with  a  good  migrating  path  upwards  in  the  next 
decade  as  the  technology  continues  to  develop. 

One  aspect  of  the  Macintosh  that  first  attracted  us  to  that  platform,  was  the  easy 
to  use  graphics  interface.  Now  that  Windows  3.0  has  come  out  for  the  DOS  operating 
system,  we  can  write  an  almost  identical  application  for  the  IBM  as  we  can  for  the  Mac, 
so  this  is  no  longer  a  concern.  Windows  3.0  running  on  a  386  allows  real  multi¬ 
tasking,  so  we  could  shrink  the  KEO  application  down  to  an  icon,  have  it  taking 
images,  and  have  a  word  processing  window  open,  taking  notes  into  a  "logbook"  file, 
or  be  graphing  an  old  image,  etc.  This  is  a  great  tool  to  have  in  a  field  instrument. 

Many  applications  are  increasingly  being  written  in  object-oriented  programs 
such  as  C++.  While  this  is  somewhat  attractive  for  us,  the  Image  Technology  board  is 
directly  supported  in  Microsoft  C.  Therefore,  to  save  the  cost  of  a  new  language,  and 
the  cost  of  learning  how  to  program  in  it,  we  think  that  development  should  start  with 
Microsoft  C. 

There  are  also  C  implementations  that  utilize  the  true  32-bit  nature  of  the  the 
386  processor.  WATCOM  8.0/386  is  a  very  good  system.  If  we  needed  to  upgrade  to 
a  true  32-bit  language.  WATCOM  is  100%  Microsoft  compatible,  so  there  is  an  easy 
upgrade  path. 

3.The  Image  Processing  Hardware.: 

There  are  three  companies  that  presently  make  image  processing  boards  for 
IBM  computers:  MATROX,  ITI,  and  CORECO.  We  have  looked  in  depth  at  MATROX 
and  ITI  as  these  most  closely  meet  our  requirements.  Both  have  their  advantages  and 
drawbacks,  which  we  will  discuss.  First  we  should  outline  how  tnese  boards  would  be 
used  and  how  this  approach  is  fundamentally  different  than  image  processing  on  the 
Macintosh. 

In  the  Macintosh  solution,  we  have  a  board  manufactured  by  Advanced 
Technologies  that  reads  the  camera  data  coming  from  the  camera  head,  directly  into 
computer  memory.  The  board  commands  all  the  basic  camera  commands  for  image 
acquisition  as  well  as  our  commands  for  the  KEO  interface  through  a  serial  interface. 
Once  the  image  is  in  the  computer,  all  processing  is  done  through  software  and 
images  are  displayed  up  on  the  computer  monitor  as  windows.  This  gives  a  great  deal 
of  flexibility  in  displays  and  processing,  but  has  the  disadvantage  of  taking  up  a  great 
deal  of  computer  memory,  and  requiring  a  lot  of  software  development  of  image 
analysis. 

The  approach  on  the  IBM  platform  is  quite  different.  The  two  image  processing 
boards  we  looked  at  have  digital  input  ports.  The  images  are  read  out  of  the  camera 
as  fast  as  the  camera  produces  the  data,  directly  into  an  image  frame  buffer  which  is 
on  the  image  processing  board.  The  image  processing  board  has  a  very  powerful 
graphics  processor  (the  34010  or  the  34020  chip),  an  arithmetic  logic  processor  for 
real-time  math  functions,  an  output  video  driver,  and  a  large  library  of  image 
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processing  routines  to  control  the  board.  Thus,  the  images  are  never  put  onto  the 
computer,  and  the  computer's  main  task  is  to  tell  the  image  processing  board  what  if  . 
next  task  is. 

This  is  advantageous  because  the  image  processing  board  has  hardware  that 
is  designed  specifically  for  these  types  of  operations  and  is  very  fast  and  efficient  at  it. 
In  addition,  this  is  the  board's  sole  responsibility,  and  the  CPU  can  be  used  for  other 
processing  applications  that  could  be  implemented  in  multi-tasking  mode  or  whatever 
Some  disadvantages  are  that  the  image  buffer  on  the  image  processing  board  is  not 
as  flexible  as  memory  in  the  computer  so  you  limit  what  you  can  do,  by  the  image 
processing  board  you  buy.  Also  the  output  display  is  not  as  flexible.  There  will  be 
applications  that  you  want  to  develop  that  are  not  capable  of  being  implemented  with 
the  standard  set  of  software  routines  provided,  and  thus  might  have  to  be  implemented 
within  the  computer  CPU.  This  involves  an  extra  step  of  reading  the  image  from  the 
image  processing  board  over  to  the  computer,  doing  the  calculation,  and  then  back 
into  the  image  processing  board  again. 

A  big  advantage  to  the  IBM  boards  are  that  they  are  compatible  with  many 
imaging  products,  from  RS-170  cameras  to  Slow-Scan  cameras.  With  these  boards, 
the  MIP  Image  Processing  workstation  could  take  data  from  many  different  sources  ail 
set  up  at  the  same  time.  One  could  envision  having  more  than  one  instrument  hooked 
up  at  the  same  time  such  as  the  MIP  CCD  camera  head  and  an  RS-170  camera  input 
coming  from  another  experimenter's  camera.  This  degree  of  flexibility  will  prove  to  be 
very  valuable  when  one  considers  that  amount  of  different  auroral  cameras  that  the  GL 
operates  alone  (film,  video,  CCD).  The  Macintosh  solution  requires  a  very  expensive 
board  designed  to  specifically  take  the  MIP  camera.  Other  sources  could  be  included 
but  additional  hardware  and  software  would  have  to  be  purchased. 

Windows  3.0  is  nearly  identical  in  use  as  the  Macintosh  operating  system,  so 
anything  that  can  be  done  on  the  Macintosh  could  be  done  on  the  IBM  now.  However, 
the  idea  of  buying  image  processing  boards  is  that  you  won't  have  to  use  the  main 
processor  for  most  tasks. 

3.1  Matrox  Image  Series  Boards  for  MIP  Application: 

Using  an  IBM  386/25MHz  PC,  we  could  implement  the  MIP  project  using  the 
MATROX  Image  Series  board  set.  This  is  an  extremely  powerful  3  board  set  that 
includes  an  advanced  display/frame  buffer  board,  a  digitizing  board  that  has  16  bit 
digital  input  port  capabilities,  and  an  advanced  Real-Time  Processing  board  to  do 
extremely  fast  mathematical  calculations.  In  addition,  a  parallel  Floating  Point 
Processor  will  become  available  in  the  near  future,  allowing  certain  extremely  fast 
computational  tools  that  would  give  the  MIP  main  frame  capabilities. 

An  advantage  that  the  MATROX  does  have  is  that  it  is  completely  system 
independent.  The  Image  Series  supports  operating  systems  such  as  UNIX,  XENIX, 
OS/2,  DOS,  and  many  types  of  languages.  We  consider  a  system  with  MS-DOS  4.01 
and  Windows  3.0  using  Microsoft  C6.0  is  used  as  this  is  parallel  to  the  requirements  of 
the  Image  Technology  board  set. 


Discussion  of  Features: 


The  MATROX  solution  would  give  us  the  most  powerful  solution  for  the  MIP  and 
indeed  a  great  number  of  its  real  strong  points  would  be  drastically  under-utilized.  The 
main  reasons  for  choosing  this  system  would  be  two-fold:  the  first  is  the  ease  and 
speed  of  implementing  our  application;  the  second  is  that  with  the  extreme  power 
available  for  image  processing,  new  ways  of  displaying  and  analyzing  data  might  be 
developed  that  were  previously  never  thought  of  due  to  lack  of  available  tools. 

Large  Capacity  Frame  Buffer 

The  Image  Series  features  a  very  large  frame  buffer.  The  MATROX  standard 
configuration  has  4  Mbytes  of  memory.  This  can  be  expanded  on  board  to  8  Mbytes. 

Image  Size  #  with  4  MB  #  with  8MB 

512x512  8  16 

256x256  32  64 

At  first  glance  this  may  not  seem  so  important.  Why  would  we  want  to  be  able  to 
display  32  256x256  images  at  once  in  our  frame  buffer?  The  answer  is  that  we  can 
store  all  sorts  of  different  types  of  information  for  the  same  image:  raw  data,  noise,  flat- 
fielding,  radar-mapping,  sorbel-edge  detection,  warping  images,  etc.  In  the  field,  we 
would  have  the  power  to  calculate  all  these  functions  and  display  them  side  by  side. 

Considering  our  own  data  set:  if  we  were  running  in  256x256  resolution  and 
were  collecting  4  images/cycle,  then  we  could  have: 

4  Raw  Images 

4  Noise 

4  Flat-fields 

4  Scratch  area  images 

4  Display  images 

=  20  256x256  images 

One  can  already  see  that  to  do  these  types  of  corrections  at  512x512  resolution 
would  require  reading  images  on  and  off  disk  for  each  image,  each  cycle.  This  would 
greatly  reduce  our  ability  to  do  many  of  these  calculations  in  real-time  in  the  field. 

Asynchronous  Digitizer 

Another  feature  that  is  very  important  to  us  is  the  digital  input  port.  The  ASD 
board  allows  either  TTL  or  RS-422  information  be  acquired  up  to  16  bits  wide.  In 
addition,  four  separate  video  sources  can  be  multiplexed  into  the  frame  buffer.  This 
board  gives  us  the  greatest  flexibility  of  input  sources  as  any  board  available  today. 
We  require  12  bits,  but  there  may  be  a  point  where  we  want  to  expand  this  to  14  bits, 
or  there  may  be  another  instrument  that  requires  16  bits  of  information  (i.e.  radar 
information  could  be  read  in  as  16  bit  data  and  stored  in  the  frame  buffer). 
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The  frame  buffer  is  the  only  one  available  that  can  be  configured  as  32  bi*s 
deep,  thus  giving  the  capability  of  very  high  precision  image  arithmetic  and  true  cole 
but  neither  of  these  are  particularly  applicable  to  the  MIP. 

Another  feature  of  the  Image  Series  which  has  potential  use  is  that  up  to  16 
ASD  boards  can  be  hooked  up  to  one  Baseboard.  This  would  give  any  PC  the  power 
to  have  many  different  sources  hooked  up  to  the  same  camera.  This  could  be  usefu 
for  us  if  we  were  to  build  more  than  one  camera  and  wanted  to  sometimes  incorporate 
a  system  with  two  cameras  (i.e.  more  filters,  one  camera  with  background  filters, 
different  types  of  cameras,  different  field-of-views,  etc.) 

Advanced  Graphics  .System  Processor  (GSP) 

The  Image  Series  uses  a  40MHz  TMS34020  GSP  chip  which  is  the  most 
advanced  available.  This  means  that  the  graphics  capabilities  of  this  board  are 
outstanding.  This  is  applicable  to  us  mainly  in  terms  of  data  analysis.  The  Image 
Series  board  set  is  ideal  for  data  analysis  where  graphics  are  overlayed  on  top  of  the 
images  for  analysis.  We  do  a  lot  of  this  type  of  work,  and  it  seems  that  our  applications 
in  the  future  will  require  more  and  more  of  this  ability.  The  34020  has  been  maximized 
for  this  kind  of  task. 

Overlay  Frame  Buffer 

The  Image  Series  board  set  has  a  separate  overlay  frame  buffer  that  is 
2Kx1  Kx4  bits.  This  means  that  the  overlay  has  no  effect  on  the  data  (even  if  we  were 
using  16  bit  data).  There  is  twice  as  much  overlay  as  the  frame  buffer  resolution 
allowing  you  to  create  complicated  graphics  applications  off-screen  and  then  pop 
them  instantly  up  on  the  screen.  This  overlay  buffer  gives  you  the  ability  to  create  a 
very  flexible  and  pleasant  looking  user  interface  using  menus,  graphics,  and  bit-maps 
with  16  color  resolution.  The  Image  Shell  comes  with  a  very  extensive  set  of  tools  to 
create  these  interfaces  including  multiple  fonts  and  text  formats.  An  X-Windows  driver 
is  presently  available  that  makes  the  overlay  look  just  like  an  X-Windows  application, 
and  a  Windows  3.0  driver  is  expected  to  be  developed. 

VGA  Switching 

A  drawback  is  that  there  is  no  VGA  mixing  available.  In  software,  one  can 
switch  back  from  VGA  to  the  Image  display  board,  so  that  if  the  operator  wanted 
access  to  the  computer  system,  or  change  parameters  under  Windows  3.0,  etc.,  you 
would  just  select  the  VGA  screen  from  software  and  then  switch  back  when  finished. 
An  application  written  for  this  board  would  not  be  implemented  this  way,  but  the 
capability  is  there  for  use  in  multi-tasking  applications  and  perhaps  very  complicated 
set  up  procedures. 

Real  Time  Processing  (RTP)  Features 

The  RTP  board  is  specifically  designed  to  execute  image  processing  functions 
in  real  time.  Perhaps  this  is  the  real  selling  point  of  this  board,  and  for  people  in  the 


image  processing  field  who  want  a  small,  affordable,  but  extremely  powerful 
processing  workstation,  the  Image  Series  board  is  clearly  unmatched  in  this  regard 

There  are  seven  custom  gate-array  chips  on  board  the  RTP  that  have  been 
especially  designed  to  maximize  the  image  processing  functions.  This  is  achieved  by 
having  the  processors  run  in  pipelined  parallel  processing.  The  RTP  board  can  add, 
subtract,  multiply,  etc.  pixels  at  rates  of  15MHz.  Thus,  all  mathematical  processes  can 
be  done  in  one  pass  through  the  image,  giving  the  ability  to  implement  these  functions 
at  video  rates.  Of  course  this  is  complete  overkill  for  us  considering  it  takes  us  at  least 
a  second  to  acquire  an  image.  What  is  more  important  though  for  this  board  is  that  in 
addition  to  the  math  capabilities,  there  is  a  Statistical  Processor,  and  a  Neighborhood 
processor  implemented  in  hardware.  There  is  also  an  arbitrary  region-of-interest  (OOI) 
hardware  implementation  which  would  allow  us  to  only  process  an  arbitrarily-shaped 
subset  of  the  image  thus  saving  processing  time. 

Statistical  Processor 

The  Statistical  Processor  allows  you  to  do  histograms  in  1  /50th  second,  and  all 
the  statistical  calculations  that  go  along  with  it,  such  as  equalizing  an  image  based  on 
the  histogram  values.  One  way  the  RIP  does  this  is  that  it  has  a  64  Kbyte  by  24  bit 
Look-Up  Table  (LUT)  that  is  used  for  real-time  calculations.  Math  functions  such  as 
multiply,  divide,  sine,  cosine,  quadratic  equations,  etc.  can  be  mapped  into  the  LUT 
and  an  image  can  be  processed  at  15  MHz  using  this  method. 

Again,  for  us.  tuis  is  complete  overkill.  We  do  not  need  histograms  at  1  '50th  of  a 
second.  But  the  64K  Statistical  LUT  has  many  applications  that  are  directly  applicable 
to  us  that  make  this  an  important  feature  to  consider.  One  immediate  application 
would  be  in  the  way  we  acquire  and  display  data.  One  would  like  to  set  up  a  system 
where  the  computer  automatically  sets  the  displays  for  the  images,  so  if  the  auroral 
activity  greatly  increases,  the  display  scale  is  automatically  adjusted.  Since  we  want 
to  display  four  images  typically  at  once  we  need  to  separately  scale  each  image  to  the 
output  image.  Ev^ry  ten  or  so  images  we  could  compute  a  histogram  and  set  up  a 
new  display  for  each  image  based  on  the  results.  This  requires  4  different  12  bit  LUT's 
for  every  cycle.  Since  a  64Kbyte  LUT  can  hold  16  12  bit  LUT’s,  many  different 
transforms  (i.e.  four  display  LUT's,  four  Multiplication  tables  for  flat-fielding,  etc)  could 
be  kept  at  all  times.  This  means  that  we  could  acquire  and  display  corrected  data  as 
quickly  as  we  could  collect  it. 

Neighborhood  Processor 

The  Neighborhood  Processor  would  allow  us  to  do  real-time  neighborhood 
processes  that  involve  convolutions  and  kernals.  These  sort  of  things  include 
sharpening,  softening,  noise  removal,  Sorbel  edge  detections,  Laplacian  edge 
detections,  grey-scale  erosion  and  dilations,  etc.  While  we  have  never  really  used 
these  functions  before,  there  may  be  cases  when  we  would.  For  instance,  if  we  are 
interested  in  studying  the  plasma  flow  along  the  edge  of  the  aurora  and  correlating  this 
with  radar  data,  it  might  be  useful  to  be  ab'e  to  display  an  auroral  arc  with  a  Sorbel 
edge  detection  on  it. 


These  are  things  that  one  would  only  develop  through  experimentation.  There 
are  not  really  any  applications  for  the  neighborhood  processor  in  the  MIP  as  we 
presently  envision  it.  However,  it  might  pay  off  in  the  long  run  to  have  such  tools 
available  to  the  scientist  should  an  application  arise. 

Floating  Point  Processor  Option  (FPU) 

The  Image  Series  will  have  the  option  of  adding  a  floating  point  processor.  The 
FPU  is  the  companion  chip  to  the  TMS34020  and  thus  is  fully  implemented  into  the 
system  architecture.  Its  true  power  lies  in  the  fact  that  it  can  run  as  in  independent 
processor  and  thus,  give  very  high  processing  speeds.  Software  for  the  FPU  option  is 
just  becoming  available.  Ideally,  we  would  want  to  be  able  to  program  our  own 
algorithms  using  this  processor,  but  this  will  not  be  practical  as  the  programming 
would  need  to  be  implemented  in  micro-code  and  would  be  too  time  consuming  to 
develop.  Instead,  the  FPU,  much  like  the  rest  of  the  board,  will  be  supolied  with  a  set 
of  routines  with  optimized  routines  for  the  FPU. 

The  set  of  routines  will  be  image  processing  oriented  such  as  transformations, 
and  FFT's.  Some  high-precision  math  functions  will  also  be  available.  An  important 
question  for  us  to  ask  is:  can  we  replace  the  Northwest  Transformation  and  implement 
it  on  the  MATROX  board  in  hardware?  If  we  could,  it  would  be  a  big  selling  point  for 
the  board.  Presently,  the  FPU  routines  will  supply  an  arbitrary  2nd  order  geometric 
warping  function  that  will  implement  in  2  seconds.  Dave  Lucas  seems  to  think  that  this 
may  be  able  to  approximate  the  Northwest  transform.  If  this  is  the  case,  the  MIP  with 
this  capability  would  be  able  to  transform  the  MIP  images  to  geographical  coordinates 
in  2  seconds  per  image,  whether  the  image  was  stationary  or  moving.  This  is  a 
incredibly  powerful  tool  not  only  as  a  field  instrument  but  also  for  speeding  of  the 
process  of  data  analysis  in  the  lab. 

The  draw  backs  of  the  FPU  are  that  it  is  not  really  feasible  to  program  our  own 
algorithms  into  it,  and  that  the  2nd  order  warping  function  is  not  really  the  Northwest 
Transformation.  So  we  would  have  to  investigate  whether  the  investment  required  for 
this  feature  is  justified. 

Frame  Buffer  Memory  Mapping 

A  final  architectural  point  of  the  Image  Series  board  that  should  be  noted  is  that 
the  frame  buffer  can  be  fully  mapped  into  the  PC's  high  map.  This  means  that  the 
4Mbytes  of  memory  could  be  mapped  into  the  PC,  say  at  memory  locations  4MB  to 
8Mbytes.  This  is  a  powerful  feature  for  implementation  of  custom  developed 
algorithms  that  run  in  software.  With  this  feature,  we  could  automatically  access  the 
data  in  the  frame  buffer  without  having  to  read  back  and  forth  between  the  frame  buffer 
and  the  computer's  memory. 


Of  course,  we  again  have  to  decide  if  this  is  really  an  important  feature  to  us.  At 
8MHz  bus  speed,  a  512x512  image  would  take  approximately  1/4  second  at  the  most 
to  read  into  the  computer's  memory.  A  256x256  image  would  only  take  a  fourth  of  that. 


MATROX  Support 


MATROX  is  a  quickly  expanding  company  with  a  long  nisto. 
image  products.  Their  company  headquarters  in  Dorval  is  very  impressive  and  th;  r 
sales  engineers  and  application  engineers  have  been  extremely  knowledgeable  , 
helpful  and  additionally  always  prompt  and  professional  in  returning  calls  and  filling 
requests  for  information.  The  Image  Series  is  in  production  mode  with  boards  at  REV 
2.0,  and  there  are  about  60  companies  with  about  400  boards  in  the  field.  MATROX  is 
always  working  on  developing  new  products  both  for  image  processing  in  general  and 
in  addition  to  further  expand  the  capabilities  of  the  Image  Series  using  the  30MHz 
Image  Bus.  I  believe  that  MATROX  would  supply  superb  technical  support  especially 
now  that  we  have  visited  the  company. 

Drawbacks  and  Thoughts 

The  first  immediate  drawback  is  that  this  is  definitely  the  most  expensive 
solution  .  One  would  really  have  to  decide  if  the  features  of  the  Image  Series  are 
worth  the  investment.  However,  comparing  the  options,  the  MAC  llci  option  would 
have  a  similar  and  not  give  nearly  as  much  power  as  the  MATROX  board.  The  costs  of 
the  MAC  computer,  and  the  customized  Photometries  board  reduce  the  cost  benefit  of 
this  option  greatly  when  compared  to  the  MATROX. 

Another  drawback  to  the  MATROX  is  that  the  output  resolution  is  1280x1024 
This  means  that  the  smallest  screen  we  could  use  would  be  a  16"  monitor  (although 
smaller  ones  might  become  available).  High  resolution  is  very  nice,  but  limits  our 
choices  in  monitors.  The  VGA  switching  scheme  is  adequate  but  a  mixing  scheme 
such  as  developed  by  Image  Technology  seems  better. 

The  MATROX  board  is  definitely  a  three  board  set.  It  is  possible  to  use  it  as  a 
two-board  set  without  the  RTP,  but  then  the  real  power  of  the  board  is  lost.  A  three 
board  set  limits  your  ability  to  miniaturize  the  system,  where  as  a  one  (ITI  VS-100-AT) 
or  two  board  (ITI  AFG)  implementation  might  give  more  flexibility  of  packaging  For. 
example  there  is  a  portable  available  that  has  a  built-in  8”  1024x768  color  monitor. 

When  investing  in  such  expensive  hardware,  we  have  to  look  at  how  much  of 
what  we  do  is  applicable  to  that  hardware  and  how  much  we  would  have  to  invest  in 
software.  Software  has  the  advantage  that  it  can  be  easily  modified,  ported  over  to 
different  systems,  and  its  performance  can  be  steadily  increased  as  hardware 
improvements  are  made.  An  example  of  this  is  the  Northwest  transformation  being 
implemented  on  the  12  MHz  286,  verses  on  a  25MHz  386,  and  even  on  a  33MHz  i486" 
We  could  spend  the  extra  dollars  on  a  high  performance  i486  rather  than  on  hardware 
that  will  always  be  the  same.  (The  advantage  of  course  of  the  hardware  is  that  it  has 
already  been  maximized  for  the  specific  application.) 

Thus,  a  strategy  needs  to  be  developed.  Do  we  want  to  invest  a  great  deal  now 
in  hardware,  or  slowly  migrate  up  with  the  evolving  technology.  The  point  should  be 
made  that  the  MATROX  hardware  does  not  limit  you  to  migrating  up  as  well,  as  we  can 
still  implement  our  applications  in  software  as  noted  in  the  above  discussion 


In  the  final  analysis,  we  have  to  decide  how  much  of  our  application  we  want  to 
develop  in  hardware  and  how  much  in  software.  We  also  have  to  remember  that  a 
particular  piece  of  hardware  does  not  restrict  us  from  developing  software,  but  makes 
the  investment  in  the  hardware  less  worthwhile  if  it  is  underutilized.  In  terms  of  a 
flexible  system  that  immediately  meets  our  needs  without  extensive  new  programming, 
etc.,  the  MATROX  is  clearly  our  best  solution,  one  that  also  clears  the  best  path  for  thp 
future  of  the  MIP  in  terms  of  choices  of  operating  systems,  powerful  tools,  and 
flexibility.  We  must  decide  how  much  we  want  to  have  such  a  system,  verses  a 
cheaper  system  that  is  not  such  a  powerful  image  processor. 


3.2  Image  Technology  Boards 


Image  Technology,  Inc.  (ITI)  offers  a  3  board  set  called  the  Advanced  Frame 
Grabber  Series  for  the  IBM.  Two  of  the  boards  are  currently  available,  and  these  are 
the  base  boards  that  we  would  need  to  control  the  MIP.  The  two  board  set  includes  a 
IK  x  IK  x  16  frame  buffer  a  16  MHz  TMS34010,  an  16  bit  ALU  with  8  bit 
Multiplier/Comparator,  12  bit  digital  input  port,  3  RS-170  inputs,  all  the  appropriate 
LUT’s,  and  a  VGA  mixing  option: 


Options: 


Advanced  Frame  Grabber  Kit  with  VGA  Option: 

VP1 510-KIT-51 2-U-C2-M3-AT 
ITEX-AFG  Subroutine  Library: 

ITEXVP1 500-A-S 

TMS  42010  Graphics  Development  Software  Kit: 
GDK-VP1 500-A-S 

Image  Processing  Accelerator  (2Mbyte): 

VP51 02-KIT-AT 
With  4  Mbytes: 

VP51 04-KIT-AT 
ITEX-IPA  Subroutine  Library: 

ITEXVP51 00-A-S 

TMS  320C30  Program  Development  Kit 


From  the  above  list,  one  can  see  there  are  many  configurations  of  this  system. 
This  is  one  of  the  advantages  to  the  ITI  board  set.  We  could  start  with  the  minimum 
configuration  which  in  most  cases  is  probably  more  than  enough  power  for  our 
application,  but  as  we  want  to  expand  the  use  of  the  MIP,  additional  boards  or  software 
can  be  added.  Additional  needs  might  be:  higher  resolution  data  runs  (processing  at 
512x512),  complicated  processing  algorithms  such  as  geometric  transformations  at 
acquisition  rates,  complicated  image  processing  to  unfold  Fabry-Perot  data. 


Let’s  step  through  all  the  configurations  and  briefly  discuss  what  the  options 


offer: 


.  Minimum  configurations: 

A  powerful  platform  for  the  KEO  camera  using  12  bit  256,  ■’  •< 

.  Adding  the  TMS  Software  KIT: 

Allows  us  to  add  on-board  routines  utilizing  the  power  c,  VivT'34 C 
tailored  to  our  specific  image  processing  needs,  getting  past  the  obsts-.i?  uawng  ro 
write  data  into  the  computer  over  the  comparatively  slow  bus.  (Noi  in  a-  m-..co  of 
issue  with  256x256  images.) 

.  Adding  the  Image  Processing  Accelerator: 

(This  does  no  longer  include  the  TMS  kit,  is  priceo  tor  a  2  MByte 
configuration,  and  includes  the  IPA  software) 

This  allows  us  to  process  images  in  real-time,  increase  the  memory 
capacity  of  the  board-set  by  two  and  would  allow  us  the  capability  to  do  histograms, 
geometric  transformations,  convolutions,  and  such  at  real-time  speeds.  In  addition,  this 
board  has  two  serial  ports,  and  a  generalized  parallel  I/O  port.  This  is  particularly 
applicable  if  we  were  to  add  a  Fabry-Perot  instrument  to  the  camera  the'  vna  need  a 
great  deal  of  processing  power,  and  additional  instrument  control. 

.  Upgrading  the  IPA  to  4  Mbytes  of  memory: 

Would  allow  us  to  process  more  images;  most  likely  not  necessary  unless 
we  wanted  to  do  a  great  deal  of  image  to  image  processing  on  512x51 2  images. 

.  Using  the  IPA  with  the  Software  Kit:  (2MB) 

Would  allow  us  to  write  custom  code  lor  the  IPA  that  was  located  on  the 
board  itself  (similar  to  the  TMS  Kit). 

.  Rolls-Royce:  AFG  with  IPA,  and  both  Software  Development  Kits: 

With  2  Mbytes 
With  4  Mbytes 

It  should  be  kept  in  mind  that  neither  the  TMS  kit  or  any  of  the  IP,4  products  are 
available  yet.  The  TMS  Kit  is  completed,  but  not  packaged  for  the  public  ye:  7  he  IPA 
hardware  is  completed,  but  the  software  is  not  yet  complete. 

Thus,  using  the  ITI  board  set,  we  have  a  large  range  of  Image  Processing 
platforms  we  can  use.  Most  of  the  research  has  been  completed,  and  this  ;s  ‘me  lowest 

cost  system. 

Hardware  Description 

The  following  outlines  the  functionality  of  the  Image  Technology  AFG  Board  Set 
from  the  input  to  the  output  stages.  Figure  1  is  the  AFG  Block  Diagram  taken  from  the 
AFG  User’s  Manual.  It  gives  a  detailed  hardware  description  and  shows  the  bit 
resolution  of  all  the  functions  on  the  board. 

The  frame  buffer  is  16  bits  deep,  so  theoretically  we  can  use  images  up  to  16  bits 
of  resolution  on  this  board.  In  practice,  though,  there  is  really  only  12  bits  of  resolution 
because  none  of  the  inputs  support  16  bits,  except  the  VISIONbus.  Theoretically,  if  we 
need  to  get  higher  resolution,  we  could  build  a  camera  interface  that  would  directly 

interface  to  this  bus. 

The  AFG  has  five  different  inputs.  Four  are  regular  RS-170  camera  inputs, 
except  that  variable  scan  video  inputs  are  also  supported  with  the  many  synch  signals 


that  come  into  the  board.  All  video  signals  are  converted  to  8  bit  deep  images,  and 
depending  on  the  board,  are  converted  to  512x480,  or  640x480.  We  would  use  the 
512x480  mode  so  that  we  could  fit  four  images  into  the  frame  buffer. 

The  most  important  input  port  to  us  is  the  Digital  Input  Port  which  supports 
images  of  any  size  (depending  on  the  incoming  clock  signals)  and  has  a  maximum 
resolution  of  12  bits.  This  is  perfect  for  our  MIP  design  and  the  camera  electronics  have 
been  made  to  be  compatible  with  this  board.  At  this  point,  the  data  can  go  directly  into 
the  Frame  Buffer  along  a  16  bit  data  path. 

The  display  memory  is  1Kx1Kx16  bits.  The  upper  four  bits  of  display  memory 
are  typically  used  for  graphics  overlays,  but  there  will  be  cases  where  we  will  want  to 
use  all  16  bits  as  real  data  (i.e.  image  arithmetic). 

There  is  an  ALU  section  (Arithmetic  Logic  Unit)  on  the  board  that  can  do  16  bit 
addition/subtraction,  barrel  shifting  and  logic  operations  at  the  full  speed  of  the  board 
(16MHz  for  an  image  and  a  constant,  and  8MHz  for  two  images).  This  means  that 
images  can  be  processed  as  fast  as  the  clock  cycle  of  the  board.  A  typical  application 
would  be  to  subtract  a  background  image  from  the  data.  For  a  256x256  image,  there 
are: 


256x256  pixels  =  65536  pixels  /  8M  pixels/sec 
=  8.2  msec/image 

For  a  512x512  image,  the  benchmark  is  38  msec/image.  This  shows  that  for  a  lot 
of  the  image  arithmetic  we  want  to  do  can  be  done  in  real  time.  This  gives  us  a 
capability  we  have  never  had  before. 

The  Processing  section  also  has  a  12  bit  LUT.  This  will  be  very  useful  to  process 
images  in  real  time  to  create  an  output  buffer.  Our  application  would  be  something  like 
this:  We  have  1/4  of  the  frame  buffer  reserved  for  the  four  256x256  images  from  the 
camera,  but  to  display  them,  we  would  want  to  display  ,hem  all  with  different  intensity 
scales.  So  for  each  image,  we  do  a  quick  statistical  analysis,  set  up  the  LUT  table  and 
write  the  image  into  the  1/4  of  the  frame  buffer  that  is  output  to  the  monitor. 

The  only  drawback  to  this  processing  section  is  that  the  multiplier  only  allows  8 
bit  inputs  (which  makes  sense  since  the  maximum  output  can  only  be  16  bits).  One 
thing  we  would  like  to  do  in  the  field  is  to  correct  the  image  for  optical  aberrations  (i.e.  a 
flat-field).  This  would  require  us  to  multiply  the  image,  by  its  appropriate  flat-fielc 
correction.  Ideally,  this  would  just  multiply  one  high  resolution  number  by  another. 
While  we  could  read  the  image  into  the  computer  and  do  it  there,  or  with  the  TMS 
software  development  kit,  you  could  write  some  code  for  the  34010  (a  32-bit  processor) 
to  do  it  quicker,  we  can  also  overcome  the  processor  limitation  and  do  a  higher 
resolution  multiplication  using  a  multi-step  method  described  in  Appendix  A.  If  we  were 
to  use  an  8  bit  flat-field,  a  256x256  multiplication  would  finish  in: 


256x256: 

512x512: 


1  /20th  second 
1  /5th  second 


For  a  16  bit  flat-field: 

256x256:  1  /6th  second 

512x512:  1/2  second 

It  may  be  seen  from  these  results  that  we  could  do  basic  image  proce  r 
functions  at  data  acquisition  rates.  This  would  also  allow  us  to  expand  tools  for  i,i  a  ;e 

processing  in  the  field. 

The  frame  buffer  is  1  Kxl  Kx16.  This  would  allow  us  to  hold: 

4  512x51 2  images 

16  256x256  images 

While  this  seems  sufficient,  there  are  a  few  limitations  that  we  have  to  cor.vc.-jf 
First  of  all,  the  new  MIP  filter  wheel  will  have  provisions  for  five  filters.  If  one  of  tnt-Se  :s 
going  to  be  a  background  filter,  then  there  are  only  four  images  of  interest,  if  we 
wanted  to  take  in  five  512x512  images,  we  couldn't  fit  these  images  into  the  frame 
buffer  without  writing  one  of  these  images  into  the  comp-t^'s  mrnmc",  .  In  add.;  o  ' .  u 
really  display  these  images  together  on  one  screen,  we  will  need  to  display  them  at 
different  intensity  scales.  To  do  this  in  256x256  mode,  we  would  create  an  area  in  tr.e 
frame  buffer  for  the  output  images  to  sit,  but  in  512x512  mode,  there  isn't  any  room 

With  this  problem  in  mind,  one  has  to  evaluate  how  important  it  is  to  have 
512x512  capability.  If  the  board  set  were  upgraded  with  an  IPA  board  as  well,  then 
there  would  be  ample  memory  available: 


IPA  with  2  Mbytes: 

8 

512x512  images 

32 

256x256  images 

IPA  with  4  Mbytes: 

12 

512x512  images 

48 

256x256  images 

Let's  discuss  how  one  could  set  up  the  frame  buffer  using  256x256  images  and 
assuming  that  only  4  filters  are  ultimately  used.  The  frame  buffer  could  be  set  up  mao  4 
distinct  areas  each  holding  four  images.  The  first  would  be  the  raw  data.  The  second 
would  hold  the  flat-fielding  or  background  data.  The  third  would  be  an  area  for  scratch 
arithmetic.  The  fourth  would  be  the  screen  buffer,  containing  the  final  images  that 
would  appear  on  the  display  monitor.  This  would  allow  us  a  very  flexible  image 
processing  system.  If  one  wanted  to  use  five  filters  and  display  five  images  on  one 
monitor,  there  are  many  ways  to  do  this,  but  we  would  have  to  think  about  the  best  way 
to  implement  this. 

The  TMS34010  is  the  main  processor  on  the  image  board  and  it  is  a  true  32-04 
CPU  designed  specifically  for  graphics  operations.  It  controls  all  communications  with 
the  host  computer  (our  386)  and  all  the  graphics  operations. 

The  display  interface  has  a  12  bit  in  8  bit  out  LUT  which  is  exactly  what  we  neuc 
for  our  images.  With  this  LUT,  we  can  do  instantaneous  pseudo-color  renditions  of 


4  4 


images,  scaling,  etc.  There  are  three  color  LUT’s  which  provide  the  color  palette  and 
finally  the  tri-DAC  video  output  to  a  high-resolution  multi-synching  monitor.  Tfie 
TMS34010  can  program  images  to  any  monitor,  so  that  we  can  have  as  many  different 
resolutions  on  the  output  as  we  have  monitors.  This  is  helpful  if  we  wanted  a  small,  low 
resolution  monitor  for  use  out  in  the  field,  out  a  large,  high  resolution  for  use  in  the  lab. 
We  could  simply  just  switch  monitors  and  set  the  board  up  for  the  new  resolu!;°n. 

The  final  piece  of  hardware  that  we  would  use  would  be  the  VGA  mixing  board 
This  would  allow  us  to  use  only  one  monitor  for  our  image  processing  workstation.  In 
traditional  image  processing,  you  typically  have  one  monitor  displaying  the  images  and 
one  acting  as  your  computer  terminal  (much  like  the  ASIP  and  Northwest  System 
operate).  To  reduce  size,  we  want  to  be  able  to  operate  using  only  one  monitor  (at 
least  in  the  field).  The  VGA  board  allows  us  to  operate  in  both  modes.  When  in  the  lab, 
if  two  monitors  was  desired,  we  could  take  out  the  VGA  board  and  use  two  monitors.  In 
the  field,  we  could  insert  the  VGA  board  and  use  only  one  monitor  for  display. 

The  VGA  board  allows  us  to  have  menus,  and  windows  over  the  images,  or  vice- 
versa.  This  would  be  very  useful  to  us,  as  we  could  make  the  application  run  from  one 
terminal  using  all  the  graphics  interface  techniques  of  the  Windows  3.0  system,  and 
have  the  images  on  the  same  screen.  While  the  software  is  running  automatically,  the 
application  icon  could  be  shrunk  in  the  corner.  When  the  operator  wants  to  change  a 
parameter,  they  would  simply  click  the  mouse  on  the  icon,  and  the  user  menus  would 
pop-up  just  like  a  normal  Mac  application.  This  would  give  us  a  consistent  and  easy 
user  interface,  that  wouldn't  have  to  be  different  depending  on  whether  you  used  a  one 
monitor  or  two  monitor  system. 

The  output  resolution  with  the  VGA  mixing  board  is  programmable  to  either 
640x480,  or  800x600.  This  provides  us  with  good  resolution  for  small  monitors  for  use 
in  the  field,  and  provides  comparable  resolution  to  that  of  the  Macintosh. 

Software  Description 

The  AFG  board  set  has  a  set  of  routines  called  the  ITEX-AFG.  This  forms  a  fairly 
complete  set  of  image  processing  routines  from  graphics  functions,  image  arithmetic, 
and  LUT  management  to  geometrical  warping  functions,  convolutions,  and 
morphological  functions.  The  software  has  been  very  carefully  evaluated  for  this 
application,  and  has  been  found  to  be  very  suitable. 

There  are  some  tasks  that  would  require  more  manipulation,  such  as  the 
multiplication  for  flat-fielding  but  this  is  hardware  limited  and  the  same  manipulation 
would  have  to  be  implemented  on  the  MATROX  system. 

Another  area  of  concern  is  transforming  12  bit  Corrected  data  into  re-scaled, 
displayable  formats  --  either  grey-scale,  or  pseudo-color.  This  would  be  implemented 
by  writing  LUT  transforms  into  and  out  of  the  INPUT  LUT  which  is  12  bits  in/12  bits  out. 
The  problem  is  that  for  every  image,  every  cycle,  an  LUT  would  have  to  be  read  in  and 
out  of  the  INPUT  LUT,  taking  up  a  considerable  amount  of  time.  We  have  yet  to  obtain 
timing  parameters  from  Image  Technology  so  this  may  not  be  a  concern.  It  is  an 


unfortunate  restriction  of  the  harowu.c  .  .i-.i  .  .  .  a,  ,,,is  by  having  a 

64Kbyte  x  24  bit  LUT,  so  that  16  LOT'S  ecu  id  \  vy. 

Functions  such  as  histogram,:.  ;.j>  -•  ...  b :  2x51 2  image 

(although  this  has  to  be  confirmed).  3 .  L,  a  LLC.^sC  ■  s-.ugram  would  be 

about  .325  seconds  which  is  much  faster  ;ba;.  we  realun.ca'.y  need 

All  the  image  processing  functions  mat  vvc  a.u  a--. enable  under  the  ITEX- 
AFG  package.  The  routines  are  not  as  fiexioie,  peweriu:.  ut  as  gu.ck  as  the  MATROX 
board  set.  but  that  seem  to  match  our  n  eeos wcm 

Summary 

The  AFG  Series  provides  us  with  the  least  expensive  solution  for  an  MIP 
workstation.  It  has  the  advantage  of  having  several  different  paths  of  upgrading  it  both 
through  new  ITI  hardware  such  as  the  iPA  uoaiu,  ana  ui.uuyi.  upg.caing  the  computer 
system  such  as  to  a  33MHz  486,  and  implementing  a  iot  of  the  functions  in  the 

computer. 

There  are  some  limitations  howevei  I  ms  Diug-.^i  oi.e  wou-c  be  the  fact  that  the 
buffer  size  is  limited.  This  is  a  problem  we  wii!  rur  into  immeu.ately  and  we  have  to 
decide  if  the  capability  of  more  than  four  512x512  images  is  important  to  us.  Other 
limitations  will  be  in  speed.  It  seems  that  trie  benchmarks  of  the  AFG  will  be  a  little 
faster  than  the  Macintosh  solution,  but  can  not  be  improved  by  upgrading  the  computer. 
However,  a  final  analysis  of  performance  verses  our  needs  will  be  completed  in  the 
near  future  to  see  if  these  are  real  issues  to  us. 

The  ITEX  routines  will  give  us  a  capability  of  experimenting  with  different  image- 
processing  techniques  such  as  Sorbei  Edge  detection  if  we  ever  wanted  to  do  such 
things.  The  AFG  is  an  excellent  compromise  between  p,;ifoirnarce  and  price. 


Appendix  4 


Flat-fielding  Scheme  with  8-Bit  Arithmetic  and  an 
alternative  16-Bit  Scheme 

Keo  Consultants  6/15/90 

With  8  Bit  Arithmetic,  we  can  still  achieve  a  flat-field  with  our  data  in  real-time 
using  the  following  method.  We  limit  ourselves  to  an  8  bit  flat-field  correction,  giving  us 
a  resolution  of  255.  We  could  further  increase  this  correction  to  4095,  if  we  wanted  to 
extend  this  algorithm  to  a  16  bit  flat-field. 

First,  let’s  define  a  flat  field.  If  we  assume  the  center  pixel  value  to  be  the 
normalized  pixel,  then  everything  is  related  to  this.  This  is  not  necessary  for  the 
algorithm,  but  it  makes  it  easier  to  picture.  Suppose  we  exaggerate  and  consider  we 
could  lose  1  /8th  of  the  signal,  due  to  vignetting  and  other  optical  losses.  We  could 
then,  for  the  sake  of  the  algorithm,  define  our  8  bit  flat-field  to  map  to  a  0  to  8 
correction.  This  gives  us: 

8/256  =  0.03125  resolution. 

Thus,  if  a  pixel  towards  the  edge  of  an  image  needs  a  correction  of  4.5  due  to 
vignetting,  image  intensifier  effect,  etc.,  the  flat  field  value  would  be: 

4.5/0.03125  =  144 

To  achieve  maximum  flat-fielding  resolution,  we  could  normalize  the  8  bit  flat- 
field  value  to  the  actual  flat-field  data: 

minimum  pixel  =  0.57/center  pixel 

maximum  pixel  =  5.60/center  pixel 

(5.60-0.57)/256  =  0.0196  resolution. 

Thus,  a  pixel  which  needs  a  correction  of  4.5,  would  have  a  flat-field  value  of: 
4.5/0.019  =  229. 

While  this  would  give  us  the  most  accurate  flat-fielding  result,  we  would  have  to 
do  a  divide  to  renormalize  the  data,  and  this  would  take  quite  a  while.  If  we  limit  our 
flat-field  normalization  to  8  or  16  (a  resolution  of  0.0625),  then  to  re-normalize  the  data, 
we  can  just  to  an  arithmetic  shift  to  the  right  of  3  or  4.  This  can  be  achieved  at  the  clock 
rate  of  the  ITI  AFG  board. 

The  following  algorithm  uses  all  the  standard  AFG  routines,  and  each  take  one 
pass  through  the  ALU  on  board,  thus  we  will  calculate  the  flat-field  time  next: 


The  flat  field  will  be  called  FF  and  is  8  bits  deep:  FF(8). 
The  data  will  be  called  D  and  is  1 2  bits  deep:  D(1 2). 

The  high  byte  of  D  is  called  D(H),  and  the  low  byte  D(L). 
Resultant  mathematical  fields  are  labeled  FI  (16),  F2(16). 

The  following  is  a  step  by  step  algorithm: 

(i)  FF  x  D(L)  =  FI 

(ii)  FFxD(H)  =  F2 

(iii)  FI  =  ASR(5)->F1 

(iv)  F2  =  ASL(3)-->F2 

(v)  FI  =  F1+F2 

Result  is  stored  in  FI  and  has  16  bits  of  real  data. 


Flere  is  an  example: 

Normalize  to  8:  Resolution  is  0.031 25 
Take  a  pixel  whose  flat  field  is  2:  2/(0.031 25)  =  64 

FF(8)  =  64  01000000 

Take  a  pixel  data  value  of  2730: 

D(12)  =  2730  0000101010101010 

Result  should  be:  2730  x  2  =  5460. 

(i)  FF  x  D(L)  =  10,880  =  FI 

0010101010000000 


(ii)  FF  x  D(H)  =  640  =  F2 


(iii)  FI  =  ASR(5)->F1  =  340 


0000001010000000 


0000000101010100 


(iv)  F2  =  ASL(3)->F2  =  5120 


0001010000000000 


(v)  FI  =  FI  +  F2  =  5460 


0001010101010100 


* 
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Each  pass  takes  38  msec  for  a  512x512  image.  Thus,  for  the  above  flat-field, 
the  process  took: 


5  x  38  msec  =  190  msec  (  or  1  /5th  second). 

This  corresponds  to  roughly  a  7MHz  pixel  rate.  Normally,  our  images  will  be 
256x256  (which  will  also  give  us  plenty  of  scrap  room  in  the  memory  of  the  AFG),  and 
thus  our  pixel  time  will  be  reduced  by  4,  so  a  flat-field  this  way  would  be  achieved  in 
about: 


Flat  Field  for  256x256:  50  msec  (  or  1  /20th  second). 

A  16  bit  flat  field  would  have  a  much  higher  resolution,  and  would  take  probably  about 
4  times  as  long. 

Note  that  we  know  as  long  as  our  flat-field  ratio  is  under  16,  we  will  never  have  a  data 
overflow,  because  our  data  is  1 2  bits,  and  the  pixel  depth  is  1 6. 

The  16-Bit  Alternative: 

Increasing  the  resolution  to  16  bits,  would  mean  needing  a  32  bit  result  for  the 
data  and  thus  increase  the  mathematics  greatly.  The  following  is  a  simplified  scheme 
using  a  16  bit  flat-field,  and  normalizing  the  flat  field  to  16: 

16/65536  =  0.00024 

The  solution  is: 

(i)  FF(L)  X  D(L)  =  FI 

(ii)  FF(H)  X  D(L)  =  F2 
(lii)  FF(L)  X  D(H)  =  F3 

(iv)  FF(H)  X  D(H)  =  F4 

(v)  FI  +  ASR(8)[F2]  =  FI'  +  CARRY  FLAG1 

(vi)  FI '  +  ASR(8)[F3]  =  F2"  +  CARRY  FLAG2 

(vii)  F4  +  ASL(8)[F2]  +  CARRY  FLAG1  +  CARRY  FLAG2  =  F4' 

(viii)  F4'  +  ASL(8)[F3]  =  F4" 

(ix)  ASR(12)[F1"]  =  F1* 

(x)  ASL(4)[F4”]  =  F4‘ 

(xi)  FI*  +  F4*  =  FT*  =  RESULT 

Time  for  each  for  this  process  adds  up  to  15  passes  through  the  processor.  For 
a  512x512  image: 

15  x  38  msec  =  570  msec  (just  over  1/2  second) 

For  a  256x256  image: 

15x9  msec  =  135  msec  (about  1  /8th  second) 


Let's  take  an  example: 

Flat  field  =  5.35  ~>  FF  =  5.35x(65536/16)  =  21913.6 
Data  =  1736, 

Result  =  9287.6 


FF: 

0101010110011010 

D: 

0000011011001000 

(i) 

FI: 

0111100001010000 

(ii) 

F2: 

0100001001101000 

(iii) 

F3: 

0000001110011100 

(iv) 

F4: 

0000000111111110 

(v) 

FI': 

1110000001010000  +  no-carry 

(Vi) 

FI": 

0111110001010000  +  carry 

(vii) 

F4’: 

0000010010000001 

(viii) 

F4": 

0000001001000100 

(ix) 

F1‘: 

0000000000000111 

(x) 

F4*: 

0010010001000000 

(x) 

RESULT: 

00100100010001 1 1 

=  9283 

Error  is:  (9287.6  -  9283)/9287.6  =  .0005  (or  .05%) 

Doing  the  same  calculation  with  the  8-bit  scheme  gives  9276,  which  gives 
about  1  1/2  times  as  much  error,  which  is  still  negligible. 


